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ETAPS Foreword 


Welcome to the 27th ETAPS! ETAPS 2024 took place in Luxembourg City, the 
beautiful capital of Luxembourg. 

ETAPS 2024 is the 27th instance of the European Joint Conferences on Theory and 
Practice of Software. ETAPS is an annual federated conference established in 1998, 
and consists of four conferences: ESOP, FASE, FoSSaCS, and TACAS. Each con- 
ference has its own Program Committee (PC) and its own Steering Committee (SC). 
The conferences cover various aspects of software systems, ranging from theoretical 
computer science to foundations of programming languages, analysis tools, and formal 
approaches to software engineering. Organising these conferences in a coherent, highly 
synchronized conference programme enables researchers to participate in an exciting 
event, having the possibility to meet many colleagues working in different directions in 
the field, and to easily attend talks of different conferences. On the weekend before the 
main conference, numerous satellite workshops took place that attracted many 
researchers from all over the globe. 

ETAPS 2024 received 352 submissions in total, 117 of which were accepted, 
yielding an overall acceptance rate of 33%. I thank all the authors for their interest in 
ETAPS, all the reviewers for their reviewing efforts, the PC members for their con- 
tributions, and in particular the PC (co-)chairs for their hard work in running this entire 
intensive process. Last but not least, my congratulations to all authors of the accepted 
papers! 

ETAPS 2024 featured the unifying invited speakers Sandrine Blazy (University of 
Rennes, France) and Lars Birkedal (Aarhus University, Denmark), and the invited 
speakers Ruzica Piskac (Yale University, USA) for TACAS and Jérôme Leroux 
(Laboratoire Bordelais de Recherche en Informatique, France) for FoSSaCS. Invited 
tutorials were provided by Tamar Sharon (Radboud University, the Netherlands) on 
computer ethics and David Monniaux (Verimag, France) on abstract interpretation. 

As part of the programme we had the first ETAPS industry day. The goal of this day 
was to bring industrial practitioners into the heart of the research community and to 
catalyze the interaction between industry and academia. The day was organized by 
Nikolai Kosmatov (Thales Research and Technology, France) and Andrzej Wasowski 
(IT University of Copenhagen, Denmark). 

ETAPS 2024 was organized by the SnT - Interdisciplinary Centre for Security, 
Reliability and Trust, University of Luxembourg. The University of Luxembourg was 
founded in 2003. The university is one of the best and most international young 
universities with 6,000 students from 130 countries and 1,500 academics from all over 
the globe. The local organisation team consisted of Peter Y.A. Ryan (general chair), 
Peter B. Roenne (organisation chair), Maxime Cordy and Renzo Gaston Degiovanni 
(workshop chairs), Magali Martin and Isana Nascimento (event manager), Marjan 
Skrobot (publicity chair), and Afonso Arriaga (local proceedings chair). This team also 
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organised the online edition of ETAPS 2021, and now we are happy that they agreed to 
also organise a physical edition of ETAPS. 

ETAPS 2024 is further supported by the following associations and societies: 
ETAPS e.V., EATCS (European Association for Theoretical Computer Science), 
EAPLS (European Association for Programming Languages and Systems), and EASST 
(European Association of Software Science and Technology). 

The ETAPS Steering Committee consists of an Executive Board, and representa- 
tives of the individual ETAPS conferences, as well as representatives of EATCS, 
EAPLS, and EASST. The Executive Board consists of Marieke Huisman (Twente, 
chair), Andrzej Wasowski (Copenhagen), Thomas Noll (Aachen), Jan Kofron (Prague), 
Barbara König (Duisburg), Arnd Hartmanns (Twente), Caterina Urban (Inria), Jan 
Křetínský (Munich), Elizabeth Polgreen (Edinburgh), and Lenore Zuck (Chicago). 

Other members of the steering committee are: Maurice ter Beek (Pisa), Dirk Beyer 
(Munich), Artur Boronat (Leicester), Luis Caires (Lisboa), Ana Cavalcanti (York), 
Ferruccio Damiani (Torino), Bernd Finkbeiner (Saarland), Gordon Fraser (Passau), 
Arie Gurfinkel (Waterloo), Reiner Hahnle (Darmstadt), Reiko Heckel (Leicester), 
Marijn Heule (Pittsburgh), Joost-Pieter Katoen (Aachen and Twente), Delia Kesner 
(Paris), Naoki Kobayashi (Tokyo), Fabrice Kordon (Paris), Laura Kovacs (Vienna), 
Mark Lawford (Hamilton), Tiziana Margaria (Limerick), Claudio Menghi (Hamilton 
and Bergamo), Andrzej Murawski (Oxford), Laure Petrucci (Paris), Peter Y.A. Ryan 
(Luxembourg), Don Sannella (Edinburgh), Viktor Vafeiadis (Kaiserslautern), Stepha- 
nie Weirich (Pennsylvania), Anton Wijs (Eindhoven), and James Worrell (Oxford). 

I would like to take this opportunity to thank all authors, keynote speakers, atten- 
dees, organizers of the satellite workshops, and Springer Nature for their support. 
ETAPS 2024 was also generously supported by a RESCOM grant from the Luxem- 
bourg National Research Foundation (project 18015543). I hope you all enjoyed 
ETAPS 2024. 

Finally, a big thanks to both Peters, Magali and Isana and their local organization 
team for all their enormous efforts to make ETAPS a fantastic event. 


April 2024 Marieke Huisman 
ETAPS SC Chair 
ETAPS e.V. President 


Preface 


This volume contains the papers presented at the 27th International Conference on 
Foundations of Software Science and Computation Structures (FoSSaCS 2024), which 
was held during April 8—11, 2024 in Luxembourg City, Luxembourg. The conference 
is dedicated to foundational research with a clear significance for software science and 
brings together research on theories and methods to support the analysis, integration, 
synthesis, transformation, and verification of programs and software systems. 

In addition to an invited talk by Jérôme Leroux (Laboratoire Bordelais de Recherche 
en Informatique, France) on “Ackermannian Completion of Separators”, the program 
consisted of 24 talks on contributed papers, selected from 79 submissions. Each sub- 
mission was assessed by three or more Program Committee members, with the help of 
external reviewers. The conference management system EasyChair was used to handle 
the submissions, to conduct the electronic Program Committee discussions, and to 
assist with the assembly of the proceedings. 

We wish to thank all the authors who submitted papers for consideration, the 
members of the Program Committee for their conscientious work, and all additional 
reviewers who assisted the Program Committee in the evaluation process. We would 
also like to thank Andrzej Murawski, the FoSSaCS Steering Committee Chair for 
various pieces of advice, and the members of the ESOP/FASE/FoSSaCS joint Artifact 
Evaluation Committee for the artifact evaluation. Finally, we would like to thank the 
ETAPS organization for providing an excellent environment for FoSSaCS, the other 
conferences and the workshops. 
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Ackermannian Completion of Separators 


Jérôme Leroux O 


Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR. 5800, F-33400 Talence, France 
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Abstract. Vector addition systems (VAS for short), or equivalently vec- 
tor addition systems with states, or Petri nets are a long established 
model of concurrency with extensive applications in modeling and anal- 
ysis of hardware, software and database systems, as well as chemical, 
biological and business processes. The central algorithmic problem is 
reachability: whether from a given initial configuration there exists a se- 
quence of valid execution steps that reaches a given final configuration. 
The complexity of the problem has remained unsettled since the 1960s, 
and was recently proved to be Ackermannian-complete. 

In 2009, we proved that the reachability problem can be decided with 
a simple algorithm by observing that negative instances of the reacha- 
bility problem can be witnessed by partitioning the set configurations 
into semilinear sets called complete separators. Since we can decide in el- 
ementary time if a pair of semilinear sets denotes a complete separator, 
the size of such a witness is Ackermannian in the worst case. 

In this paper, we show how recent results about the reachability problem 
can be combined to derive a matching upper-bound, i.e. for every nega- 
tive instance of the reachability problem, we can effectively compute in 
Ackermannian time a complete separator witnessing that property. 


1 Introduction 


Vector addition systems [8] (VAS for short), or equivalently vector addition sys- 
tems with states [7], or Petri nets are one of the most popular formal methods 
for the representation and the analysis of parallel processes [3]. The central algo- 
rithmic problem is reachability: whether from a given initial configuration there 
exists a sequence of valid execution steps that reaches a given final configuration. 
Many important computational problems in logic and complexity reduce or are 
even equivalent to this problem [2216]. 


After an incomplete proof by Sacerdote and Tenney [20], decidability of 
the problem was established by Mayr [17/19], whose proof was then simpli- 
fied by Kosaraju [9]. Building on the further refinements made by Lambert 
in the 1990s [IO], in 2015, a first complexity upper-bound of the reachability 
problem was provided [12] more than thirty years after the presentation of the 
algorithm introduced by Mayr [90]. The upper-bound given in that paper is 
“cubic Ackermannian”, i.e. in Fs (see [21]). This complexity bound was obtained 
by analyzing the Mayr algorithm. With a refined algorithm and a new ranking 
© The Author(s) 2024 
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function for proving termination, an Ackermannian complexity upper-bound was 
obtained in [15]. This means that the reachability problem can be solved in time 
bounded by F.,(p(n)) where p is a primitive recursive function and where F,, is 
an Ackermann function. Very recently, this complexity bound was proved to be 


optimal [42]. 


While the complexity of the reachability problem is settled, its parameterized 
version, in fixed dimension d, is still open with a large complexity gap between the 
lower-bound and the upper-bound. Some recent results provided ways to decrease 
that gap (see for instance [IMI]) but the problem remains open. Since there 
exists d-dimensional VAS with finite but very large reachability sets [I8], any 
reachability algorithm directly based on the Mayr algorithm will necessarily fail 
in providing a better complexity upper-bound. In fact that algorithm enumerates 
in some way each possible reachable configurations when the reachability set is 
finite. 


There is another algorithm for deciding the reachability problem indepen- 
dent of the Mayr algorithm. In fact, in |13|, we introduced a simple enumerating 
algorithm for deciding the reachability problem by observing that negative in- 
stances of the reachability problem can be witnessed by partitioning the set of 
configurations into semilinear sets called complete separators. Since we can de- 
cide in elementary time if a pair of semilinear sets denotes a complete separator, 
and the reachability problem is Ackermannian-hard, the size of such a witness 
is necessarily Ackermannian in the worst case. 


In this paper, we take the opportunity to show how to combine papers [I5] 
and to prove that from any negative instance of the reachability problem, we 
can effectively compute in Ackermannian time a complete separator witnessing 
that property. This result prove the optimality of algorithms based on complete 
separators for deciding the general reachability problem. Since this paper is an 
invited paper at FOSSACS’24, so without any reviewing process, no new proof 
are given in this paper. If a proof is given, it just to be self-content. But in any 
case, those proofs are copy-past from [I5] and [I3]. 


Even if our result does not provide a better understanding of the complexity 
of the parameterized reachability problem, it shows that algorithms based on 
complete separators are optimal in general dimension. 


2 Basic Notions 


In this section, we introduce basic notions and notation. 


Notation for Vectors of Integers. By Z we denote the set of integers, and 
by N the set {0,1,2,...} of non-negative integers. Given d € N, the elements 
of Zt are called (d-dim) vectors; they are denoted in bold face, and for x € Z4 
we put x = (x(1),...,x(d)) so that we can refer to the vector components. In 
this context, d is called the dimension of x. We use the component-wise sum 
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x + y of vectors, and their component-wise order x < y. For c € N, we put 
c- x= (c-x(1),...,c- x(d)). 


Linear and Semilinear Sets. A set L C N? is linear if there are d-dim vectors 
b, the basis, and p1, ..., Pk, the periods (for k € N), such that L = {x € N° | 
x = b+u(1)-pı +- -+u(k)- Pr for some u € N*}. In this case, by a presentation 
of L we mean the tuple (b, p1,.--, Pk). 

A set S C N° is semilinear if it is a finite union of linear sets, i.e. S = 
Lı U---U Ly where L; are linear sets for all j. In this case, by a presentation 
of S we mean the sequence of presentations of L;,...,L,. When we say that a 
semilinear set S is given, we mean that we are given a presentation of S; when 
we say that S is effectively constructible in some context, we mean that there is 
an algorithm computing its presentation (in the respective context). 

We recall that a set S C N? is semilinear if, and only if, it is expressible in 
Presburger arithmetic [4]; the respective transformations between presentations 
and formulas are effective and elementary. Hence if S C N? is semilinear, then 
also its complement, denoted as S, is semilinear, and S is effectively constructible 
when (a presentation of) S is given. 


Fast Growing Functions. The Grzegorczyk hierarchy [5J16] is defined thanks 
to a family (Fa)aen of functions Fa : N > N such that every primitive recursive 
function is asymptotically bounded by some function Fy. This family is defined 
by Fo(n) = n+ 1 and inductively by Fu4i(n) = F? (n) for every n,d € N. 
Observe that Fı(n) = 2n +1, Fo(n) = 2°*1(n +1) — 1, and F3(n) grows as a 
tower of n exponentials. It follows that F3 is a non elementary function since 
it eventually exceeds any fixed iteration of the exponential function. An Acker- 
mannian function, denoted as F, is defined thanks to the diagonal extraction 
F.,(n) = F,,41(n) for every n € N. This function is non primitive recursive. 


Vector Addition Systems. A (d-dim) vector addition system (VAS for short) 
is a finite set A of vectors in Z% called actions. Vectors x € N? are called 
configurations, and with an action a we associate the binary relation & on the 
configurations in N by putting x & y for all x,y € N? such that y — x =a. 
The relations + are naturally extended to the relations 5 for finite sequences 
o =a,...a, of actions by x S y if x ~>--- 4 y for all x,y € N°. 


On the set N? of configurations we define the reachability relation A". we 


put x all y if there is ø € A* such that x 4 y. For x € N? and X C N? 


we put POSTA (x) = {y € N4|x 2, y}, and Post (X) = Uxex POST’ (x). 
Symmetrically, for y € N? and Y C N? we put PREX (y) = {x € N¢| x A, y} 


def 


and PREX (Y) = Uyey PREA (y). By X £, Y we denote that x “> y for some 
xe€Xandyey. 

The semilinear reachability problem takes as input a triple (X, A, Y) where 
X, Y are (presentations of) semi-linear sets of configurations of a VAS A, and 


checks if X “> Y hold. In the standard definition of the reachability problem the 
sets X, Y are singletons; the problem is decidable [I9], and it has been recently 
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shown to be Ackermann-complete [15[14)2]. It is well-known (and easy to show) 
that the above more general version (the semilinear reachability problem) is 
tightly related to the standard version, and has thus the same complexity. 


3 Separators 


A separator is a negative instance of the semilinear reachability problem, i.e. a 
triple (X, A, Y) where X, Y are semilinear sets of configurations of a VAS A 


such that =(X nan Y). The domain D of a separator (X, A, Y) is the semilinear 
set XUY. Notice that X, D, and Y forms a partition of Nt. When the domain 
is empty, the separator is said to be complete. Notice that a triple (X, A, Y) is 
a complete separator if, and only if, (X,Y) is a partition of N? into semilinear 
sets such that y — x Æ a for every x € X, y € Y, and a € A. In particular this 
property is decidable in elementary time by encoding it as the satisfiabibility of 
a Presburger formula. A separator (X’, Y’) is called a completion of a separator 
(X,Y) if (X’, Y’) is complete, X C X’ and Y C Y’. 


In we proved that every separator can be effectively completed. In this 
paper, we show how this result can be extended with optimal complexity bounds. 
More formally, we prove that any separator can be completed in Ackermannian 
time. The Ackermannian lower-bound is immediate since the reachability prob- 
lem for VAS is Ackermannian-complete and as already mentioned, we can check 
in elementary time if a pair of semilinear sets is a completion of a separator. The 
most difficult part of the result is the Ackermannian upper-bound. 


4 Semi-Pseudo-Linear Sets 


Given two semilinear sets X, Y of configurations of a VAS A, the sets POST’ (X)N 
Y and PRE4(Y)/M X are not semilinear in general. However, we proved in [13] 
that those sets are semi-pseudo-linear, a class of sets that can be tightly over- 
approximated by semilinear sets called linearizations. Linearizations are obtained 
by solving several instances of the semilinear reachability problem. Since in j 
we provided an Ackermannian upper-bounds on that decision problem, we can 
reasonably think that the completion of separators can be done in Ackermannian 
time. To prove that result, in this section we provide complexity bounds on the 
size of linearizations. Those linearizations will be used in the next section for 
completing separators in Ackermannian time. 


Let us recall some definitions. A monoid M is a set of configurations such 
that 0 € M, and such that M +M C M. The monoid spanned by a set P C N? 
is the set of finite sums of vectors in P. It is denoted as XP. A vector a € N? 
is called an interior vector of a monoid M, if for every m € M, there exists a 
natural number n > 1 such that na € m + M. 


A pseudo-linear set is a set X C N? such that there exists a linear set 
L = b+ M where M is the monoid spanned by the periods of L, such that 
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X C L and such that for every finite set R of interior vectors of M, there 
exists x € X such that x + XR C X. In that case, the linear set L is called a 
linearization of X. A semi-pseudo-linear set X is a finite union of pseudo-linear 
sets X = X, U... U Xx. In that case a semilinear set of the form Lı U... U Lk 
where L; is a linearization of X; is a called a linearization of X. 


By combining the proof of [13] Theorem 6.4] with [I5], we deduce the fol- 
lowing theorem where fa is a function of the form Fai3(Cn) for some constant 
C independent of d. In this theorem, the size in binary or in unary does not 
change the result and there is a lot of freedom in the definition of the size of 
presentations of semilinear sets and VAS. 


Theorem 1. Given two semilinear sets X and Y of configurations of a d-dim 
VAS A, the sets POST’ (X)NY and PRE’ (Y)NX are semi-pseudo-linear. More- 
over, we can effectively compute in time fa(n) where n is the size of the input, 
presentations of linearizations of those sets. 


The tightness of linearization approximations can be emphasis by introducing 
the notion of rank] given in [13]. Formally, the rank of a set X C N¢, denoted as 


rank X is the minimal r € {—co,0,...,d} such that there exists a semi-linear set 
S that contains X of the form bı + M, U... U bg + My, where M,,..., Mz, are 
monoids spanned by at most r vectors. In [I3], we prove that rank(X) = —oo iff 


X is empty, rank(X) < rank(Y) if X C Y, and the following theorem. 


Theorem 2 (Proposition 7.10 of [13]). Let S1,S2 be linearizations of two 
non-empty semi-pseudo-linear sets X 1, X2 with an empty intersection. We have: 


rank(Sı N S2) < rank(Xı U X2) 


5 Ackermannian Completion 


We show in this section who a separator (X, A, Y) can be completed in Acker- 
mannian time. We follow the algorithm introduced in by first proving that if 
(X, A, Y) is not complete, i.e. if the domain D is non empty, we can effectively 
compute a separator (X’, A, Y’) with a domain D’ such that X C X’, Y CY’, 
and such that rank(D’) < rank(D). It follows that by applying at most d times 
this algorithm where d is the dimension of A, we get a complete separator. 


Let n be the size of the separator (X, A, Y). 

The set Y’ is obtained as follows. Since D is semilinear and effectively com- 
putable in elementary time, it follows from Theorem [1] that we can compute in 
time fa(E(n)) where E is some fixed elementary function a linearization U of 
the semi-pseudo-linear set POST% (X) N D. We introduce Y’ = Y u (D \ U). 

Let us prove that (X, A, Y’) is a separator. By contradiction, assume that 


X “4 Y’. Since «(X “5 Y), and Y’ = Y U (D \ U), we deduce that X &5 


‘In this notion is called dimension but in our context, the dimension word is 
already used for the number of components of a vector. 
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(D \ U). However, since POSTA (X) MD C U we get a contradiction. Hence 
(X, A, Y’) is a separator and its domain is equal to DN U. 

The set X’ is obtained symmetrically. Since D N U is semilinear and effec- 
tively computable in elementary time, it follows from Theorem [I] that we can 
compute in time fa(£"(fa(E(n)))) where E’ is some fixed elementary function a 
linearization V of the semi-pseudo-linear set PREX (Y’) A DN U. We introduce 
X’=xu((DNU)\V). 

Symmetrically, we deduce that (X’, A, Y’) is a separator and its domain D’ 
is equal to DN UNV. 


Since (X, A, Y’) is a separator, it follows that POST’ (X) and PRE% (Y’) have 
an empty intersection. In particular the semi-pseudo-linear sets POST% (X) MD 
and PREA (Y’) 1 DNU have an empty intersection. If one of those semi-pseudo- 
linear sets is empty then D’ is empty and in particular rank(D’) < rank(D). 
Otherwise, from T heorem[2}we deduce that the rank of UNV is strictly bounded 
by the rank of the union of POST’ (X)ND and PREA (Y’)NDNU. Since this set is 
included in D, and D’ is included in UNV, we deduce that rank(D’) < rank(D). 


By replacing E and E’ by E + E’, we can assume without loss of generality 
that E = E’. By iterating the previous construction at most d times, we deduce 
that from any separator (X, A, Y) of size n, we can compute in time (fao F)?4(n) 
a completion of it. We deduce the main theorem of that paper. 


Theorem 3. Separators can be completed in Ackermannian time. 


6 Conclusion 


In this paper, we have shown that separators can be completed in Ackermannian 
time. Our computation is based on a generic algorithm given in Section |5| This 
algorithm can be implemented as soon as we have an oracle computing semilin- 
ear sets over-approximating the sets POST’ (X)MD and PRE, (Y) MD. If those 
approximations are not linearizations, the termination of the algorithm is no 
longer true in general. However, since its correctness is maintained, it should be 
interesting to benchmark such an algorithm when using heuristics for implement- 
ing oracles computing reachability set over-approximations (based on abstract 
interpretation, acceleration techniques, parameterized invariant, and so on). 
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Abstract. We consider two-player games over finite graphs in which 
both players are restricted by fairness constraints on their moves. Given 
a two player game graph G = (V, E) and a set of fair moves Ey C E a 
player is said to play fair in G if they choose an edge e € Ey infinitely 
often whenever the source node of e is visited infinitely often. Otherwise, 
they play unfair. We equip such games with two w-regular winning con- 
ditions a and @ deciding the winner of mutually fair and mutually unfair 
plays, respectively. Whenever one player plays fair and the other plays 
unfair, the fairly playing player wins the game. The resulting games are 
called fair a/8 games. 

We formalize fair a/ games and show that they are determined. For 
fair parity/parity games, i.e., fair a/ games where a and 8 are given 
each by a parity condition over G, we provide a polynomial reduction 
to (normal) parity games via a gadget construction inspired by the re- 
duction of stochastic parity games to parity games. We further give a 
direct symbolic fixpoint algorithm to solve fair parity/parity games. On a 
conceptual level, we illustrate the translation between the gadget-based 
reduction and the direct symbolic algorithm which uncovers the underly- 
ing similarities of solution algorithms for fair and stochastic parity games, 
as well as for the recently considered class of fair games in which only 
one player is restricted by fair moves. 


Keywords: games on graphs, fairness, two-player games, parity games 


1 Introduction 


Omega-regular games are a popular abstract modelling formalism for many core 
computational problems in the context of correct-by-construction synthesis of 
reactive software or hardware. This abstract view was initiated by the seminal 
work of Church [8| and its independent solutions by Biichi and Landweber and 
Rabin [18J5]. Since then these ideas have been refined and extended for solving 
the reactive synthesis problems {17|20/14). 
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However, before using any such synthesis technique, the reactive software de- 
sign problem at hand needs to be abstractly modelled as a two-player game. In 
order for the subsequently synthesized software to be ‘correct-by-construction’ 
this game graph needs to reflect all possible interactions between involved com- 
ponents in an abstract manner. Building such a game graph with the ‘right’ 
level of abstraction is a known severe challenge, in particular, if the synthesized 
software is interacting with existing components that already possess certain be- 
havior. Here, part of the modelling challenge amounts to finding the ‘right’ power 
of both players in the resulting abstract game to ensure that winning strategies 
do not fail to exist due to an unnecessarily conservative overapproximation of 
modeling uncertainty (or the dual problem due to underapproximation). 

In this context, fairness has been adopted as a notion to abstractly model 
known characteristics of the involved components in a very concise manner. Fair- 
ness assumptions have been used in model checking [I] and scheduler synthesis 
for the classical AMBA arbiter or shared resource management [6]. Notably, 
fairness assumptions have also gained attention in cyber-physical system design 
215M1] and robot motion planning [92]. In all these applications, fairness is 
used as an assumption that the synthesized (or verified) component can rely 
on. In particular, if these assumptions are modelled by transition fairness over 
a two-player game arenq>| (W, Va, E) —ie., by a set of fair environment moves 
Eş C E (i.e., with Vy as their domain) that need to be taken infinitely often 
if the source node is seen infinitely often along a play — the resulting synthesis 
games can be solved efficiently E9]. 

While most existing work has only looked at fairness as an assumption to 
weaken the opponent in the synthesis game, all mentioned applications also 
naturally allow for scenarios where multiple components with intrinsically fair 
behavior are interacting with each other in a non-trivial manner. For example, 
the ability of a concurrent process to eventually free a shared resource might 
depend on how fair re-allocation is implemented in other threads. On an abstract 
level, the formal reasoning about such scenarios requires to understand how the 
interactive decision making of two dependent processes is influenced by intrinsic 
fairness constraints imposed on their decisions. Algorithmically, these synthesis 
questions require fairness restrictions on both players in a game, i.e., do not 
restrict the domain of fair moves Eş to one player only. We refer to such games 
simply as fair games. 

Motivating Example. In order to better illustrate the challenges arising from 
solving such fair games, consider two robots in a shared workspace with narrow 
passages between adjacent regions that only one robot can pass at a time. One 
robot (say the green one) has an w-regular objective a that specifies desired 
sequences of visited regions in the workspace. The other (red) robot tries to 
prevent the green robot from achieving this sequence. In order to rule out trivial 
spoiling strategies of the red robot, both robots need to implement a tie-breaking 


3 Whenever we interpret players in a one-sided manner as environment and system, 
we choose the environment player as the V-player, as we need to take all possible 
environment moves into account. Similarly, the system is the 4-player in this scenario. 
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mechanism for obstacle avoidance, i.e., they must eventually move left or right 
if an obstacle blocks their way. 

Now consider the scenario where both robots are facing each other at a gate, 
as depicted in Fig. |1| While both robots block the gate from one side, neither 
of them can move forward, but if the green robots moves left or the red robot 
moves right, the other robot can take the gate to reach region A. With the 
mentioned requirement for tie-breaking, none of the robots is allowed to block 
the gate forever and both eventually have to move to the side. 


Fig. 1: Deadlock caused by fairness constraints of two robots facing a door. 


Now let us assume that region A is important for both robots, hence, both 
robots have an incentive to enter region A first, to then move the game to an 
area preferable to them. However, the robot who breaks the tie first, (i.e., fulfills 
its fairness condition first) allows the other robot to enter region A first, which 
gives both robots the incentive to behave unfair. While it is very intuitive to 
make a player lose when she plays unfair and the other player plays fair, it is 
unclear who wins the game if both players play unfair. 

To resolve this issue, we can make the objectives of the robots completely 
adversarial by assigning one of the players (say, green) the winner in a play where 
both players play unfair. In the above example, this would give the red robot the 
incentive to break the tie first. While this makes it harder for the red robot to 
spoil the objective of the green one, we might be interested in a more symmetric 
game which does not favor the green robot in all non-determined states of the 
graph. We therefore consider a second w-regular objective 8 that determines the 
winner of (mutually) unfair plays. This results in fair games G = (A, a, 3) which 
are determined (as shown in Sec. B). 


Contribution. Motivated by the above mentioned examples where interactive 
decision making of two dependent processes is influenced by intrinsic fairness con- 
straints imposed on their decisions, this paper studies fair games G = (A, a, 8) 
as their abstraction. In particular, we give solution algorithms for these games 
when both a and £ are parity conditions induced by two different priority func- 
tions over the node set. We call such games fair parity/parity games. 
Obviously, the previously discussed one-sided version of fair games, which 
we call V-fair games (as only the V-player (i.e., the environment) is restricted 
by strong transition fairness), is a special case of fair games. Both enumerative 
[9] and symbolic solution algorithms [4] have recently been proposed for V- 
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fair games, showing that strong transition fairness can be handled efficiently 
in both types of algorithms. This observation is closely related to a result for 
stochastic games, i.e., two-player games with an additional ‘half’ player that 
takes all its moves uniformly at random. For the purpose of qualitative analysis, 
such stochastic parity games have been shown in [7| to be reducible to (standard) 
parity games by the use of “gadgets” that turn stochastic nodes into a small 
sequence of V- and 4-player nodes. While it is known that stochastic games can 
be reduced to V-fair games (and hence, fair games), it was not investigated how 
the different solution approaches compare. The main conceptual contribution of 
this paper is a unified understanding of all these solution approaches for the 
general class of fair games. 

Concretely, our contribution is three-fold: 
(1) We formalize fair games as a generalization of V-fair games and stochastic 
games such that they are determined. 
(2) We show a reduction of fair parity/parity games to (standard) parity games, 
inspired by the gadget-based reduction of stochastic parity games to parity games 
in [7]. This reduction enables the use of parity game solvers over the reduced 
game (in particular enumerative ones such as Zielonka’s algorithm [24]) and gives 
a gadget-based reduction of V-fair parity games to parity games as a corollary. 
(3) We then show how our gadget construction can be used to define a symbolic 
fixpoint algorithm to solve fair parity/parity games directly (without the need 
for a reduction). We show the direct symbolic algorithm for V-fair parity games 
in [4] coinciding with our algorithm for this particular subclass of fair games. 

With this, we believe that this paper uncovers the underlying similarities 
of solution algorithms for fair, V-fair and stochastic parity games. Further, we 
show how these conceptual similarities can be used to build both enumerative 
and direct symbolic algorithms. This is of interest as both are known to have 
complementary strengths, depending on how the synthesis instance is provided, 
and this connection was, to the best of our knowledge, not known before. 

All omitted proofs are available in the extended version of this paper [10]. 


2 Preliminaries 


We introduce infinite-duration w-regular two-player games over finite graphs 
with additional strong transition fairness conditions on both players. For read- 
ability, we call the considered games (and their respective notions) simply fair. 


Infinite Sequences. We denote the set of infinite sequences over a set U by 
U”. We often view sequences T = u1u2... E U” as functions 7 : N > U, writing 
T(i) = ui. Furthermore, we let Inf (T) := {u € U | Vi. 3j > i. 7(7) = u} denote the 
set of elements of U that occur infinitely often in 7. Given a function f : U > W, 
we denote by f(r) € W® the pointwise application of f to r. Given a natural 
number n, we write [n] := {1,...,n}. 


Fair Game Arenas. A fair game arena A = (V3, W,£E, Ep) consists of a set 
of nodes V = V3UVW that is partitioned into the sets of existential nodes V3 and 
universal nodes Vy, together with a set EF C V x V of moves that is partitioned 
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into the set Ey C E of fair moves and the set E\ Ey of normal moves. If Er = 0, 
then we sometimes omit this component for brevity. Given a node v € V and a 
binary relation R C V xV, we write R(v) to denote the set {w € V | (v,w) € R}. 
We assume that F is right-total, that is, E(v) #0 for all v € V. We call a node 
v fair, if it is the source node of a fair edge, i.e., E¢(v) Æ Ø and collect all fair 
nodes in the set V" = {v € V | Ey(v) # Ø} and define V” = V \ V®" to be the 
set of nodes that are not fair (‘normal nodes’). We denote Vf" = Vi" A V3 and 
yfir ea y fair N W. 


Plays. A play T = vov ... on Ais an infinite sequence of nodes s.t. vi41 E€ E(v;) 
for all 2 > 0. Given a play T = vovı..., we define the associated sequence of 
moves Tm = (vo, V1) (V1, v2). ... Additionally, if i is a player in {3,V}, we denote 
the other player by 1 — i. We let plays( A) denote the set of all plays on A. 

For a player i € {3,Y}, a play 7 is i-fair if for all nodes v € V; N Inf(7) holds 
that Eşle C Inf(tm), where Ef|ly = {(v,v’) € Ey | v € V} denotes the set of 
fair edges that start at v € V. Given a play T, we write fair;(7) to indicate that 
T is î-fair. We call a play mutually fair if it is both J3- and V-fair and mutually 
unfair if it is neither J- nor V-fair. 


Strategies. A strategy for player i € {5,V} (or, an i-strategy) is a function 
p:V*-V; > V where for each u-v € V*- V; it holds that p(u - v) € E(v). A 
strategy p is called positional if p(u - v) = p(w - v) for all u,w € V* and v E€ Yj. 

A strategy p for player i is said to admit a play T = vov,... if for all k € N, 
up E Vi implies p(vo ... Uk) = Vk+1. Alternatively, 7 is said to be compliant with 
p. We write X for the set of 4-strategies and J for the set of V-strategies. Starting 
from a node v € V, any two strategies s € X and t € I induce a unique play 
play,,(s, t) in the game arena. If we do not care about the initial node of the play, 
we simply write play(s, t). 

A strategy for player i € {5,V} is an i-fair strategy if every play it admits is 
i-fair. We write 0" (resp. I7*'") for the set of 3-fair (resp. V-fair) strategies. 


Omega-regular Winning Conditions. We consider winning conditions given 
by an w-regular [22]13] language p C V“ over the node set V. In particular, we 
write y = L and y = T to denote the trivial winning conditions @ and V’, 
respectively. In particular, we focus our attention to parity winning conditions. 
For a priority function A : V — [k] that maps nodes of a game arena to the 
natural numbers bounded by k for some k € N, the Parity(A) condition is given 
via y = {r € V® | max(Inf(A(7))) is even}. 


Omega-regular Games. An w-regular game is traditionally defined via a 
tuple G = (A,a) where A is a game arena without fair edges, i.e. Ey = Ú and 
a C V” an w-regular winning condition. An 4-strategy s € X is said to be 
winning (for 3) from a node v € V, if for all t € ZZ, play,(s,t) € a. Dually, 
a V-strategy t € IT is said to be winning (for V) from a node v € V, if for all 
s € X, play, (s,t) ¢ a. In w-regular games, every node v € V is won by one and 
only one of the players [12]73]. This property of a game is called determinacy, 
and w-regular games are determined. We denote the nodes from which Ẹ (resp. 
V) has a winning strategy in G by Win3(G) (resp. Winy(G)). When G is clear 
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from the context, we drop the parenthesis and write Wina and Winy instead. 
Determinacy then amounts to Wing U Winy = V and Wina Winy = Í. 


Node Conventions for Figures. Throughout this paper, in all figures, the 
rectangular nodes represent V-player nodes and the nodes with round corners 
represent J-player nodes. 


3 Fair Games 


As already outlined in the motivating example in Sec. [1| the interpretation of 
winning conditions over fair games influences the characteristics of resulting 
winning strategies. To formalize this intuition, we will first recall a particular 
subclass of fair games, namely those where only one player is restricted by an 
additional fairness condition, in Subsec. We will then use these games to 
motivate winning semantics for the general class of fair games. 


3.1 Determinacy of V-Fair Games 


A V-fair game is a tuple G = (A,a) where A is a game arena with V® C W 
(called a Y-fair game arena), and a is an w-regular winning condition. 

In V-fair games, fairness constraints typically model known behavior of exist- 
ing components that the 4-player (i.e., the to be synthesized system) can rely on. 
This is formalized by defining that the 4-player wins a V-fair game with winning 
condition a from node v if 


s € X. Yt € I". play, (s,t) € a. (la) 


That is, 3-player (or shortly, 3) wins if they have a strategy that can win against 
all V-fair V-strategies. 

Our intuition tells us that this can be converted to reasoning about general 
strategies for V-player (or shortly, V) by allowing 4 to win whenever V plays 
unfairly. In order to see this, we can look at the complement of Eq. {la}, i.e., 
the description of when V wins; namely, Vs € X.3t € II". play,(s,t) ¢ a. 
We can replace the quantification over fair strategies with a quantification over 
all strategies but require that, in addition to refuting a, the resulting play be 
fair: Vs € YY. at € I. fairy(play,,(s,t)) A play, (s,t) € a. Indeed, as we show in 
the extended version of this paper [IO] App. A - Lem. 2], if strategy t € H 
satisfies fairy(play,,(s,t)) then we can find a fair strategy t/ € Hf" with which 
play,,(s,¢) is compliant. This V-fair strategy would also stop s from winning. Due 
to determinacy of w-regular games, we know that the last condition is equivalent 
to dt € .Vs € X. fairy (play,,(s,t)) A play,,(s,¢) € a. In particular, this implies 
that t is fair. We conclude that the complement of Eq. (1) is the following 
equation: 


at € IT". Ys € X. play, (s, t) £ a. (1b) 


This statement is equivalent to the determinacy of V-fair games: either J-player 
has a winning strategy or V-player has a winning V-fair strategy, and the two 
cannot be true simultaneously. 
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3.2 From V-Fair Games to Defining Determined Fair Games 


Given a fair game arena A and an w-regular objective a, a natural attempt to 
define winning regions in fair games would be to generalize Eq. to 


v € Wing if Js € X". vt € M". play, (s,t) € a, and (2a) 
v € Winy if 3t € H". Ys € X°", play, (s, t) Z a. (2b) 


However, in this case, Wing U Winy 4 V. Indeed, equations and (2b) 
are not complements of each other, that is, 


ds € Sve c I!" play(s,t) € a £ Vt € H", Js € Si play(s, t) € a. 


This observation makes a fair game in which winning regions are defined via Eq. 
undetermined. The undetermined nodes O C V — nodes from which none of the 
players has a fair winning strategy — form a separate partition of nodes, i.e., 
V = Wina U Winy U O. To see this, consider the following example. 


Example 1. Consider the fair game arena depicted in Fig. [2] where fair edges are 
shown by dashed lines, a = Parity(X) and each node is labeled by its priority 
assigned by À. We observe that the existential player cannot enforce reaching 
the even node with a i-fair strategy from the two middle nodes. Every J-fair 
strategy s has a counter V-fair V-strategy: choose the fair edge outgoing from 
the square node after s chooses the fair edge outgoing from the node with round 
corners. On the other hand, the universal player cannot prevent the play from 
reaching the even node with a V-fair strategy from these nodes for exactly the 
same reason. Hence, the middle two nodes are neither in Wina nor in Winy. That 
is, these two nodes are undetermined; therefore they form O. 


= > =- 


Winy Wina 


Fig. 2: A simple fair game arena discussed in Ex. 


In order to better understand the distinction between Equations Pa| and 
we rely again on translation to w-regular games. Consider the following refor- 


mulation of Eq. (2a): 


s € LNt € IL. faira(play,,(s, t)) A (fairy (play, (s, t)) => play,,(s,t) € a). (8a) 


Similarly, the following is a reformulation of Eq. (2b): 


te WNs € Sfairy(play,,(s,t)) A (fairs(play,,(s,t)) = play,(s,t) Za). (3b*) 


From determinacy of w-regular games, the negation of the latter is: 


ds € St € Ifairy(play,,(s,t)) => (fairs(play,,(s,t)) A play, (s,t) € a). (3b) 
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We formally prove the equivalences of Eqs. and and Eqs. and 
Bb) 


in [10]. It is not hard to see that the difference between Eq. and Eq. ( 

is in the way fairness is handled. Namely, in Eq. (3a) 3 loses whenever she plays 
unfairly regardless of how V plays. Dually, in Eq. (8b) 3 wins immediately when 
V plays unfairly regardless of how J plays. It follows that determinacy can be 
regained by deciding the winner of the four different combinations of fairness 
with an w-regular winning condition each, as summarized in the following table. 


faira(7)|—>fairs(7) 
fairy(T) TE a] TE 
afairy(rT)} TE Od | TEB 


With this generalization, we obtain if 8 = y = L and ô = T, and if 
y=LandB=6=T. 

We note that the discussion of determinacy has crucial importance to the 
analysis of games and the decision of how to model particular scenarios. For ex- 
ample, if fairness of V-player arises from physical constraints (as, e.g., in [4]) then 
it might make sense to consider Eq. (2b), which corresponds to 8 = T. Dually, if 
fairness of 4-player must be adhered to, then it makes sense to consider Eq. (2a), 
which corresponds to = L. Our formulation allows to further fine tune what 
happens when both act unfairly by adjusting (. 

Given the intuition that fairness constraints are actually additional obliga- 
tions for both players, the choice of y = L and 6 = T assumed in Equations (2)- 
is very natural. However, allowing mutually unfair plays to be decided by a 
different w-regular winning condition 3, allows games with more symmetric win- 
ning semantics e.g., by setting 6 = a. We therefore restrict our attention in this 
paper to fair games with two winning conditions a and ( while if 7-player plays 
fairly but (1 — 7)-player plays unfairly, i-player wins, i.e., y := L and ô := T. 
This is formalized next. 


Definition 1 (Fair Games). A fair game G = (A,a,{3) consists of a fair 
game arena A together with two (w-regular) winning conditions a, 8 C plays(A) 
where a and 8 determine the winner of mutually fair and mutually unfair plays, 
respectively. In fair games, a play that is i-fair and (1—1)-unfair is won by player 
i. Formally, in the fair game G = (A,a, 8), v € Wing if and only if, 


ds € X.Vt € I. fairs(play,,(s,t)) A (fairy(play,,(s, t)) => play,,(s,t) € a) 
V (-faira(play,,(s, ¢)) A fairy (play, (s, t¢)) A play,(s,t)€ 8) (4) 
The determinacy of fair games follows trivially from the formulation. It follows 


that the complement of Eq. is the V winning region, defined symmetrically 
by v € Winy if and only if 


Jt € IVs € X. fairy(play,,(s, t)) A (faira(play,,(s, ¢)) = play,,(s,t) Z a) 
V(afairy (play,,(s,t)) A afairs(play,,(s, t)) A play,,(s,t) Z 8) 


Notation. We call a fair game G = (A,a,{) a fair a/8 game. Further, if œa or 
B are given by mentioned winning conditions(e.g. a = Parity(A), 6 = L), with 
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slight abuse of notation, we refer to the game with the name of the objectives 
(e.g. fair parity/L game). 


Remark 1. Stochastic games allow for an additional set V, of stochastic game 
nodes that belong to neither 3 nor Y, and for which the stochasticity is resolved 
uniformly at random. It is known that for purposes of qualitive analysis (i.e., the 
computation of almost-sure winning strategies), stochastic games can be seen as 
the special case of V-fair games in which E(v) C Ey holds for all stochastic nodes 
v € V;, and Ey N E(v) = @ for all non-stochastic nodes v € V3 U WY, that is, all 
stochastic edges are fair edges, but no non-stochastic edges are fair edges. This 
encoding treats stochastic branching as adversarial for the system (-player). 


3.3 Mutually Fair Strategies in Fair Parity Games 


In Subsec. [3.2] and in particular in Ex. [I]we have discussed the mutually unfair 
plays and strategies that take such plays into account in fair a/6 games. In this 
section, we start restricting our attention to fair parity/8 games (as this will be 
our focus for the rest of the paper) and discuss the particularities of mutually 
fair strategies in such games. We will do this with the help of the games G1 — G4 
depicted in Fig. }3} No mutually unfair plays exist in any of these games. This is 
because on all given arenas the unfair behaviour of one player makes the play 
trivially fair for the other. Therefore, the winning regions are independent of 2. 

In game G1, both nodes are won by J. V-player loses node 3 since taking the 
self loop on 3 makes the play visit 3 infinitely often, however, it forces V to play 
fairly, implying that they must take the edge to 4 infinitely often. Therefore, any 
V-fair play is won by J since the priority 4 is seen infinitely often. Also note that 
if V-player decides not to play fairly, they immediately lose since all plays are 
trivially J-fair. The trivial winning d-strategy is depicted by red edges. 

To get to game G2, we append node 1 to the left of G1. Here, all the nodes 
are won by V. This is because V-player wins node 3 by eventually taking the 
outgoing edge to 1 and then staying in 1 forever with the self-loop. By doing so 
V evades his obligation to take the fair edges by forcing each play to see node 3 
a finite number of times. One winning V-strategy is depicted by blue edges. 

To get to game G3, we append node 5 to the right of game G1. Again, all 
the nodes are won by V even though this time he cannot evade taking his fair 
edges. In this game V wins due to the obligation of J to play fairly. In a play 
starting from 3, V must eventually take the outgoing edge to 4. From there on, 
the play will visit node 4 infinitely often, forcing J to take his outgoing edge to 5 
infinitely often. As a consequence, in every mutually fair play 5 is seen infinitely 
often. Therefore, the game is won by V. A winning V-strategy is depicted by 
blue edges on the figure, with the interpretation that blue edges from node 3 are 
taken alternatingly (in every sequence). 

Finally, to get to game G4, we append two nodes to game G3. This time, all 
the nodes are won by d. J-player still needs to take their fair outgoing edges to 5 
(and this time, also to the new node 1) infinitely often. But this time she can also 
take the outgoing edge to 6 infinitely often and thereby win the game. A winning 
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J-strategy is depicted by red edges on the figure, again with the interpretation 
that red edges from node 4 are taken alternatingly (in every sequence). 


Gx Ge nO 


( ) 


HRO 
Gs: $ 


Fig. 3: Four fair parity/(@ games: dashed lines represent fair edges. Games Gi and G4 
are won by i-player and Gz and G3 are won by V-player. In each case, a respective 
winning strategy is shown by colored edges. A set of colored edges represents a strategy 
that takes only the colored edges in the game, and whenever a source node is visited 
all its colored outgoing edges are taken alternatingly. 


4 Reduction to Parity Games 


In this section, we show how fair parity games can be reduced to parity games 
without fairness constraints. We show that there is a linear reduction to parity 
games in the case that a is a parity objective and 6 = T or 8 = L; for the case 
that 6 is a non-trivial parity objective, we show that there still is a quadratic 
reduction. Our reductions work by replacing each fair node in the fair game with 
a 3-step parity gadget. This construction is inspired by the work of Chatterjee 
et al. [7] where the qualitative analysis of stochastic parity games is reduced to 
solving parity games. 

We give the formal reduction for fair parity/_ games in Subsec. [4-1]and ex- 
tend it to fair parity /parity games in Subsec. The extended version contains 
a discussion of the reduction for a restricted case of fair parity/ games (fair 
Büchi/L games), which can serve as a hand-holding introduction to the section. 


4.1 Reduction of Fair Parity/1l Games 


Let G = (A, Parity(A), L) where A = (V3, W, E, Es) is a fair game arena, V = 
Valu YW and A: V —> [2k] is the priority function. 

The reduction to parity games replaces fair nodes v € V“ in G with the 
gadgets given in Fig. Nodes v € V#"" in G are replaced with one of the 
gadgets on the top (i.e. the incoming edges to v are redirected to v in the root, 
and the outgoing edges on the third level lead to E(v) and Ey(v), which are the 
outgoing edges and outgoing fair edges of v in G, resp.) and nodes v € Vf in G 
are replaced with one of the gadgets at the bottom. The gadgets on the left are 
called existential gadgets and the ones on the right are called universal gadgets, 
referring to the player picking the first move. Nodes in V" are not altered. 

Even though the proof works for all combinations of the gadgets (i.e. one can 
replace each v € Vf?" (v e Vi") with any of the gadgets on the top (bottom)), 
due to space constraints we give the intuition only for the existential gadgets. 
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Imagine all v € Vf are replaced with their existential gadgets. Within a 
subgame that starts at a fair node v € V", the two players intuitively interact 
as follows. The 4-player gets to pick a number i, indicating the priorities (2i — 1 
or 2i) they intend to visit infinitely often in any play that visits v infinitely often. 
In turn, V-player gets to either pick an outgoing edge at v (for this, he pays the 
price of seeing the even priority 2i), or allow 4 to pick an outgoing edge (in 
which case he is rewarded with a visit to the odd priority 2i — 1). Depending 
on the owner of v, the edge picked by V (if v € V£"), or the edge picked by 3 
(if v € Vf) is required to be contained in Ey. Thus Y can insist on exploring 
fair edges at Vf nodes, but has to pay a price for it; dually, V eventually has 
to allow J to explore the fair edges at Vf nodes to win. 

In the full reduced game defined formally in the proof of Thm. [1] below, the 
owner of a fair node v can fairly win from v by either avoiding v from some 
point on forever, or eventually allowing the opponent player to explore all fair 
edges leading out of that node. The owner wins by playing unfairly if and only 
if the opponent also plays unfairly and the owner is the V-player. 


Fig. 4: Existential (left) and universal (right) gadgets for v € VĒ" (top) and v € Vf" 
(bottom) in fair parity/1 games. For i € [1,k+ 1], priorities of nodes v? and vř are 


given below them, priorities of nodes v; are ignored, and the priority of v is unaltered. 


Theorem 1. Let G = (A, Parity(A),-_L) where A = (V3,W,£E, Es) is a fair 
game arena, V = V3U VW and X: V > [2k] is the priority function. Then there 
exists a parity game G” on the node set V’ with V C V’ and |V’| < n(3k + 3) 
over 2k + 1 priorities such that for i € {3,V}, Win;(G) = Win;(G’)NV. 


Proof (Sketch). Let G’ = (V3, W, E’, Q : V’ — [2k + 1]) be the parity game 
obtained by replacing the fair nodes in G with an arbitrary combination of 
their corresponding existential and universal gadgets in Fig. |4| Let V” = V5U 
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W = V U V&d where V represent the nodes coming from G and V*4 represent 
the nodes coming from the gadgets. Note that the maximum priority in G” is 
MaXodd = 2k + 1 which comes only from the gadget nodes ved The maximum 
even priority in G’ is maxXeyen = 2k which can come from both V8*4 and V. It is 
easy to see that |V’| < n(3k + 3) and G” uses priorities [2k + 1]. To prove the 
correctness, we recall that the winning regions for fair parity/__ games are given 
via Eq. (Bah, i.e. v € Win3(G) if and only if 


ds € Yt € I faira(play,,(s,t)) A (fairy(play,,(s,t)) => play,(s,t)€ a). Bal 


(=) We will first show, v € Win3(G’) O V = v € Wina(G). To do so, we 
will take a (positional) winning 4-strategy s’ in G” and construct an 4-strategy 
s in G such that s is J-winning in G i.e., s realizes Eq. (3a). That is, for a play 


p in G that starts from v and compliant with s Eq. (3ats) holds. 


faira (p) A (fairy(p) > p € a) (8a}s) 

For this we will show the two parts of the conjunction separately. We will 

show (i) fairs(p), ie. s € Xf", (ii) fairy(o) > p € a, i.e. every V-fair play 
compliant with s is 4-winning w.r.t. the parity condition. 


Construction of the s’-subgame G‘,,: Let s’ be a positional 4-strategy win- 
ning every play from v in G”. We will denote the subgame of G” where 3 nodes 
have only the outgoing edges u + s’(u) by G”, and call it the s’-subgame. Recall 
that all plays that start from v in G”, are 4-winning. 


Notation of n, and succ(u): For the existential gadgets for both V£" and V$, 
we call the index of the unique successor of u in G, ny. That is, s’(u) = un,- 
For the same gadgets, we will denote s'(up ,) with succ(u). For the universal 
gadgets for both V4" and Vi", we will let ny denote the index of the rightmost 
child of u that is sent to its right child by s’. That is, n, is the largest index i 
such that s’(u;) = uj. For the same gadgets, we will denote s' (uz) with succ(u). 
Construction of s: We define s : V* - V3 — V as follows. For u € Va 1. If 
Nu = k + 1, we set s(u) = succ(u). 2. Otherwise, s(u) cycles through the set 
{succ(u), E(u)} starting from succ(u). For u € Va \ V£", we set s(u) = s' (u). 
Constraining G’,, with ną: Here we will constrain G’, to its subgame by 
limiting the choices of V-player from a u replaced by the universal gadget. For 
every universal gadget encountered in G’, we limit the choices of u € Vi" to 
only u > Un, and u —> Un, +1 (if it exists). So, we remove all the other branches 
of u out of G1. We call the remaining game LG",, standing for limited G". Note 
that as LG’, is a subgame of G’, it is still S-winning. 


J-extension: Let p be some play in G compliant with s. We define a play p’ 
that is called the 4-extension of p = utu? ... as follows: p' is the play on LG", 
that follows p while ‘prioritising existential nodes on succ(u)’. What is meant 
by this is, for a ut € V®"", if ut! = succ(u"), then p’ takes the unique branch in 


LG", that leads to u’t! while passing through an existential node (u’);. That 
is, regardless of which gadget u’ is replaced by, p’ takes the branch 


= — suce(u) = uit? (branch 1) 


ub 


ut > Un => (u) 
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On the other hand if utt! Æ succ(u'), then p’ takes the only other branch in 
LG',, that is (branch 2) is taken as 


1. If ut € V®" is replaced by an J-gadget, then uf > ui, > (ui Fes aie 


v itl 
nytt >u a 


2. Ifu’ € V“ is replaced by a V-gadget, then u’ > ui, 4, > (ut) 


Note that these branches do not leave out any possible transition in p. That’s 
because 1. all the successors of a Vf" node are covered by one of the branches 


since (branch 2) leads the universal node COM , or (ui) ;+1 Which can pick 
any successor of u’. 2. all the successors of a Vf" node are covered by one of 
the branches, since by construction of s, all the successors of ut in p are in the 
set {succ(u')} U E(u"), where (branch 1) covers the succ(u’) successors, and 
(branch 2) covers the E(u’) successors since in this case the universal node 
ivV iny ; à i 

(u'), , or (u'), , 4, can pick any fair successor of u’. 

For ut 4 Vir, p' just takes uf > uitt. 

So p’ is well defined, and is a play in LG’, that starts from v. Thus, p’ is 
J-winning. Observe that if we remove the gadget nodes from p’, we get p. That 
is, the restriction of p’ to V, p’ |v= p. 


(i) faira(~): Observe that for any p in G compliant with s, by construction of s, 
the only nodes u € Vi?" that p may not be fair on, are those for which nu = k+1. 
So we only need to show that such nodes are seen only finitely often in p. Since 
plv = p’, that is equivalent to showing such a u cannot be seen infinitely often 
in its 4-extension, p’. If it is seen infinitely often in p’, then regardless of the 
gadget u is replaced with, the branch u > ug41 > ug 41 1s evoked infinitely 
often, signalling the largest priority 2k + 1. Therefore, p’ is won by Y-player, 
giving a contradiction. Therefore, we conclude p is 3-fair. 


(ii) fairy(o) > p € a: Let p be V-fair. Look at the 3-extension p’ of p. Let m be 
the largest (even) priority in Inf(p’). Due to p’ |y= p, all we need to show is the 
existence of a u € Inf(p’ |y) that has priority m. Then it automatically implies 
that the maximum priority in Inf(p) is m, and thus p is 4-winning. 

We will proceed with proof by contradiction and assume that the priority m 
appears only in V84 N Inf(p’). Now let F be the subgame of LG‘, that consists of 
nodes and edges taken infinitely often in p’. Then, priority m appears in V&A F. 
These gadget nodes must exist in F due to nodes 


—ueéV" replaced by existential gadgets, and with nu = m\2 (which corre- 
sponds to (branch 2)-1), or 

— u € V*'" replaced by universal gadgets, and with nu = m\2 — 1 (which 
corresponds to (branch 2)-2) 


Note that for all such nodes u, (branch 1) of u is also in F. This is because 
u — succ(u) is taken infinitely often in p. For u € Vf, this is due to the 
construction of s, for u € Vf", this is due to p being V-fair (remember, in this 
case succ(u) € Ep(u)). 
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Next, we will remove from F all priority m gadget nodes (and everything 
reachable only from those nodes). That is, we will prune out (branch 2) of all 
the nodes that bring in m priority gadget nodes to F. Due to the remaining 
(branch 1)s, this pruning does not cause any dead-ends. Let’s call this pruned 
subgame of F, H. Observe once more that all plays in H are 4-winning. However, 
the maximum priority in H is m—1. This is due to the remaining (branch 1)s of 
the pruned nodes having this priority. This implies that all infinite plays starting 
in H get trapped in a subgame H’ of H that doesn’t have nodes with priority 
m —1. Since non of the nodes in V“ N H’ cause a gadget node with priority m, 
non of their branches get pruned. That is, all nodes in H’ have the same outgoing 
edges in H’ and in F. Then any play that start in H’ in F, does not leave H’, 
making H’ exactly the set of nodes seen infinitely often in p’, i.e. H’ = F. This 
contradicts our initial assumption that maximum priority seen infinitely often 
in p’ being m; therefore proving p is 4-winning. 

The proof of direction (<) is similar to the proof of (=), and can be found 
in detail in the extended version |10]. 


Remark 2 (Reduction of parity/T games). In the gadgets from Fig. 4} in order 
to play unfairly from a v € vas J-player has to take its rightmost branch 
and signal priority maxogg, whereas to play unfairly from v € We, V-player 
has to take the rightmost branch and signal MaXeven. Since MaXodg > MAXeven, 
this dynamic ensures mutually unfair plays are V-winning. The gadgets for a fair 
parity/T game with A : V — [2k] can be constructed as follows with the addition 
of priority 2k +2: Take the gadgets from Fig. |4| In the existential gadget for V2?" 
add another branch + vý}; — Es(v) to vey1 and in the universal gadget for 
Vi" add a rightmost branch > vz42 > Uk > Eș(v). In the existential gadget 
for Vf" add a rightmost branch + vk+1 > vp}ı > Ey(v) and in the universal 
gadget for Vf" add another branch + vz,, > Ep(v) to Vk+1- 

All the newly added gadget nodes have priority 2k+2 and therefore MaXeven = 
2k+2 > maxodg = 2k+1, which ensures that mutually unfair plays are J-winning. 
The correctness of the construction follows as a corollary of the reduction of fair 
parity/parity games given in the next section. 


4.2 Reduction of Fair Parity/Parity Games 


In this section, we present a quadratic reduction from fair parity /parity to parity 
games. So let G = (A, Parity(A), Parity(’)) where A = (V3,W, E, Ef) is a fair 
game arena with V = V3 U Ķ and priority functions À : V > [2k], 2: V —> [d]. 

The reduction is based on ideas from the previous section, in particular adapt- 
ing the basic structure of the introduced gadgets. However, in order to correctly 
treat mutually unfair plays according to the additional parity objective I’, we 
annotate game nodes v € V with two memory values p € |d] and b € {3,V}. 
The former is used to store the maximal priority according to I that the play 
has recently seen; this value is signalled (and reset after signalling) from time 
to time in the reduced game. The value b is used to decide (at certain nodes) 
whether the memory value is signalled, or not. 
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2k+1 $=4 
+1 2k+2+p b=V 


E(v,p,b) E's(v,p, b) E(v,p, b) Es(v,p, b) E(v, p/1, 5) 


Fig. 5: Gadget for v € Vf in fair parity/parity games; u abbreviates (v, p, b). 


It indicates the player that has last taken the rightmost branch in the gadget 
for one of its fair nodes. If this bit keeps flipping between 3 and V forever, then 
both players intuitively insist on keeping control in one of their respective fair 
nodes, enabling a mutually unfair play; in the reduced game, the memory content 
p is signalled (and then reset to 1) whenever the value flips from Y to 3. 


Fig. 6: Gadget for v € Vf" in fair parity/parity games; u abbreviates (v, p, b). 


Formally, the reduction is as follows. The game is based on the set V x [d] x [1] 
of base nodes, where we use [1] to denote {3, Y}; intuitively, a node (v, p, b) from 
this set corresponds to v € V, annotated with memory values p and b as described 
above. In order to succinctly refer to the combination of taking a move in G and 
updating the memory components, we overload notation and put 


E(v, p,b) = {(w,p’,b) € V x |d] x [1] | w € E(v) and p' = max(p, T (v))} 
Es(v,p,b) = {(w,p’,b) € V x [d] x [1] | w € Ep(v) and p' = max(p, F (v))} 
for (v, p,b) € V x [d] x [1]. Thus a triple (w, p’, b) is contained in E(u, p, b) if there 


is a move (v,w) € E and p’ is the maximum of the previous memory value p 
and the current priority T (v) at v; in E(u, p, b), we require (v, w) € Ey instead. 
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In both functions, the argument b is used to explicitly set this component of the 
memory to either 3 or Y. The reduced game consists of subgames that start at 
annotated nodes u = (v,p,b) € V x [d] x [1]. In case that v € V", the game 
just proceeds according to E(v,p,b), with ownership of (v,p,b) determined by 
whether v € V3 or v € W; this corresponds to taking a move at a normal node 
in G, but updating the memory component p, and keeping the component b 
without modifying it. For fair nodes v € Vf", the subgame consists of three 
levels, and after these three steps leads back to a node from V x |d] x [1]. Fig. 
and |6|show the subgames that start at (v, b, p) € V x [d] x [1] such that v € VP" 
and v € ver, respectively, adapting the existential gadget for v € Vf and the 
universal one for v € yeer, 

The rightmost branches in these gadgets overwrite the last component b with 
J and V, respectively. The colored values in the right-most branch in the Fig. 
gadget depend on the value of b. If b = V (corresponding to V-player being the 
one that last has taken the right-most branch), then the priority 2k + 2 + p 
is signalled and the memory value p is reset to 1; if b = 4 (corresponding to 
J-player having taken the right-most branch last), then the priority 2k + 1 is 
signalled and the memory value p does not change. 


Theorem 2. Let G = (A, Parity(A), Parity([)) where A = (V3,W, E, Es) is a 
fair game arena, V = V3 U WĶ and A: V —> [2k] and T : V = |d] are priority 
functions. Then there exists a parity game G’ with 6nd(k+2) nodes and 2k+2+d 
priorities with set V x |d] x [1] of base nodes such that for all v € V, 3-player 
wins v in G if and only if 4-player wins (v,1,4) in Œ. 


Proof (Sketch). We construct the parity game G” following the above descrip- 
tion, using the gadgets from Fig. [5]and [6] to treat fair nodes. The detailed con- 
struction and the correctness proof can be found in the extended version [10]. 


We obtain the following bound on strategy sizes for fair parity/parity games. 


Lemma 1. Let G be a fair parity/parity game on n nodes. Then for both players 
the memory requirement of winning strategies in G is at most n?-n”. Furthermore, 
for each player a family of fair parity/L games (Gn)nen exists such that for all 
n, every winning strategy for the respective player requires memory at least 2”. 


Proof (Sketch). For the upper bound, we note that in a winning i-strategy for 
a fair parity/parity game, as constructed in the proof of Thm. |2| the nodes in 
V; \ Vi" have strategies with quadratic memory, but the nodes in Vf may 
have to traverse all their fair successors, and possibly one more successor. In the 
worst case, this requires an additional local memory of |E¥(v)| +1 < n for each 
VE yrr and causes an exponential blowup in the overall memory required. 
For the lower bound, we consider the case for J-player; the result for V-player 
is obtained by switching the player’s roles. Define the family (Gn)nen of games 
by letting Gn (for n € N) have exactly n+1 nodes, one node x owned by V-player 
and n nodes y; owned by J-player; let there be an edge from z to any node y; and 
two fair edges from any node y; back to x. Let all nodes have priority 0. Then 
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any winning J-strategy in Gn necessarily is J-fair. There is a fair d-strategy s 
that uses one bit as local memory for each node y; € Vf", and therefore uses 
memory of overall size 2”. The claim follows since there is no 4-fair strategy that 
uses less memory than s, which is shown by induction on n. 


5 Fixpoint Characterization of Winning Regions 


In this section, we will characterize the winning regions in fair games with 
parity conditions by means of fixpoint expressions. Thereby we provide an al- 
ternative, symbolic route to solve such games, rather than by reducing to parity 
games. We start by briefly recalling details on Boolean fixpoint expressions. 


Fixpoint expressions and fixpoint games. Let U be a finite set, let o be a 
fixed number and let f : P(U)° + P(U) be a monotone function, that is, assume 
that whenever we have sets X;, Y; C U such that X; C Y; for all 1 < 7 < o, then 
f(X%1,...,Xo) C f(M%,..., Yo). Then f and o induce the fixpoint expression 


e= NoXo. No-1X 0-1: eee .V Xo. uX. f(X, tae , Xo) (5) 


where 7; = v if i is even and n; = p if i is odd. We define the semantics 
of fixpoint expressions using parity games. Given a fixpoint expression e, the 
associated fizpoint game Ge = (W3, Wy, E, Parity(«)) for the priority function 
k : W3 U Wy > [o] is the following parity game. We put W3 = U x {1,...,0}, 
Wy = P(U)°. Moves and priorities are defined by 


E(v,i)={Z € W | ve f(Z)} k(v,i)=i 
E(Z) = {(v,i) | v € Zi} k(Z)=0 


for (v,i) € W3 and Z = (Z1, ..., Zo) € Wy. Then we say that v € U is contained 
in e (denoted v € e) if and only if J-player wins the node (v, 1) in Ge. 


Remark 3. The above game semantics for fixpoint expressions has been shown to 
be equivalent to the more traditional Knaster-Tarski semantics [8]; the cited work 
takes place in a more general setting and therefore uses slightly more verbose 
parity games. 


Next we present a fixpoint characterization of the winning regions in fair 
games of the form G = (A, Parity(A),-L) where A = (V3, Wọ, E, Ef) is a fair 
game arena, V = V3 U VW and A: V => [2k] a priority function. To be able 
to write fixpoint expressions over such games we define monotone operators on 
subsets of V by putting 


OX ={vEV | E(v)nxX FO} X={veEV| E(v) Cc xX} 
OsX ={v EV | Ev) NX FO} X ={veEV|E;(v) CX} 


for X C V and also put Cpre(X) = (VaNOX)U(WNOX). Then Cpre(X) is the 
set of nodes from which 4-player can force the game to reach a node from X in 
one step. Also, we define C; = {v € V | A(v) = i} for 1 < i < 2k. 
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Using this notation, we define a function parity : P(V)?* > P(V) by putting 


parity(X1,..., X2x) := (C1 N Cpre(X1)) U... U (Cp Cpre(Xoz,)) 


for (X1,..., Xo.) C P(V)?*. This function is monotone and it is well-known 
(see e.g [23]) that the fixpoint induced by parity characterizes the winning region 
in parity games with priorities 1 through 2k. This formula will still apply to 
‘normal’ nodes V” in the fixpoint characterization of fair parity games. 

We follow the gadget constructions from Fig. |4| (using their existential ver- 
sions) to define the following additional functions. For 1 <i < k, put 


Apres(X;, Xi41) =OX;,NOf¢Xi41  Aprey( Xi, Xig1) = Of Xi NOX, 


encoding nodes (vř, 2i) for v € VÍ and v € Vi", respectively (here, Apre 
stands for alternative predecessor function, as it encodes the additional V-choice 
of whether a fair edge is to be taken). Then, we let Ip, = {i | i odd,p <i < 2k} 
denote the set of odd priorities that lie between p and 2k, and put 


fair _ o Apres 


A,p 
Vier, Apres 


User, Aprey 
Vier, Aprey 


Using this notation, the winning region for the existential player in fair 
parity/ games with priorities 1 through 2k can be characterized by the fix- 
point expression induced by 2k + 1 and the function y that is defined to map 
(X1,...,;Xon41) E P(V)?**! — P(V) to the set 


Xi, Xi+ı 
Xi, Xi+ı 


U 0 Xorg p is odd 
U 0 Xenq1 UOf Xp p is even, 


Xi, Xi+ı 
Xi, Xi+ı 


p is odd 
oer — 


n A aN 


UUX, pis even 


v(X1,..-,Xen41) =(V" N parity) U 
(yer N U CG n git) U 


€[2k+1] 


(ern U Ci n pei) 


i€[2k+1] 


The function x therefore treats normal nodes from V" in the same way as nodes 
in standard parity games are treated, but for fair nodes with priority 7, the 
functions $$) and (7) are used to encode the respective gadget construction. 
The full fixpoint expression then is 


e= HX?2k+1- VXok. HX?2k—-1 woe VX. uXı. x(Xı, soe , X2k+1) (6) 


The first result of this section is that the fixpoint expression (6) characterizes 
the winning region of 4-player in fair parity/L games. 


Theorem 3. Let G = (A, Parity(A),-L) where A = (V3,W,£E, Es) is a fair 
game arena, V = V3 UV and A: V — [2k] is the priority function. Then the 
fixpoint expression given in (6) characterizes Win3(G). 
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Proof (Sketch). The proof is by mutual transformation of winning strategies in 
G and in the semantic game Ge for (6). The full proof can be found in [I0]. 


We note that for V-fair parity games (V/?"" = Ø), Eq. (6) instantiates to the 
fixpoint characterization given in [4]; it follows that the parity game reductions 
from Sec. [4] apply to the one-sided fair parity games considered in [4] as well. 

For fair parity/parity games, we obtain a similar fixpoint characterization, 
encoding the reduction to parity games presented in Subsec. [4.2] along the lines 
of Figures|5]and|6| Here, all involved functions work over (subsets of) the set V x 
|d] x [1] of base nodes, consisting of game nodes that are annotated with memory 
values. The definition of the fixpoint expression for fair parity/parity games 
is straight-forward but somewhat technical since the updating and resetting 
mechanisms for the memory values have to be accommodated. For brevity, we 
refrain from elaborating the required notation and the full fixpoint expression 
here, and state just the main result that yields a symbolic fixpoint algorithm for 
fair parity/parity games; full details can be found in the extended version [IO]. 


Theorem 4. Let G = (A, Parity(A), Parity([)) where A = (V3,W,E, Es) is 
a fair game arena, V = V3 UV andr: V > [2k], T : V > |d] are priority 
functions. Then there is a fixpoint expression over V x |d] x [1] with alternation 
depth 2(k +1) + d that characterizes Win3(G). 


Proof (Sketch). Again the proof is by mutual transformation of winning strate- 
gies in G and in the semantic game Ge for the fixpoint expression. The full proof 
can be found in the extended version [10]. 


6 Conclusion 


We introduce two-player games with local transition-fairness constraints for both 
players, allowing two objectives a and 8 to decide the winner of plays in which 
both players play fair and both players play unfair, respectively. We show the de- 
terminacy of this class of games in the case that a and 8 are w-regular objectives. 
In the special case that both a and £ are parity conditions, there is a reduction 
to standard parity games with blow-up quadratic in the number of priorities 
used by a and 8; if 6 = T or 6 = L, the reduction becomes even linear. We 
present both enumerative and symbolic methods to realize this reduction; in the 
process, we also obtain an exponential tight bound on the memory required by 
winning strategies in fair parity/parity games. We expect that the central idea 
behind the reduction generalizes from parity objectives to more general settings 
such as fair games in which a and 8 are Rabin, Streett, or even Emerson-Lei 
conditions, but leave this issue for future work. 
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Abstract. Stochastic two-player games model systems with an environ- 
ment that is both adversarial and stochastic. The environment is modeled 
by a player (Player 2) who tries to prevent the system (Player 1) from 
achieving its objective. We consider finitary versions of the traditional 
mean-payoff objective, replacing the long-run average of the payoffs by 
payoff average computed over a finite sliding window. Two variants have 
been considered: in one variant, the maximum window length is fixed and 
given, while in the other, it is not fixed but is required to be bounded. For 
both variants, we present complexity bounds and algorithmic solutions for 
computing strategies for Player 1 to ensure that the objective is satisfied 
with positive probability, with probability 1, or with probability at least p, 
regardless of the strategy of Player 2. The solution crucially relies on a 
reduction to the special case of non-stochastic two-player games. We give 
a general characterization of prefix-independent objectives for which this 
reduction holds. The memory requirement for both players in stochastic 
games is also the same as in non-stochastic games by our reduction. 
Moreover, for non-stochastic games, we improve upon the upper bound 
for the memory requirement of Player 1 and upon the lower bound for 
the memory requirement of Player 2. 


Keywords: Stochastic games - Finitary objectives - Mean-payoff - Reac- 
tive synthesis 


1 Introduction 


We consider two-player turn-based stochastic games played on graphs. Games are 
a central model in computer science, in particular for the verification and synthesis 
of reactive systems [18, 11,17]. A stochastic game is played by two players on 
a graph with stochastic transitions, where the players represent the system 
and the adversarial environment, while the objective represents the functional 
requirement that the synthesized system aims to satisfy with a probability p 
as large as possible. The vertices of the graph are partitioned into system, 
environment, and probabilistic vertices. A stochastic game is played in infinitely 
many rounds, starting by initially placing a token on some vertex. In every round, 
if the token is on a system or an environment vertex, then the owner of the vertex 
chooses a successor vertex; if the token is on a probabilistic vertex, then the 
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successor vertex is chosen according to a given probability distribution. The token 
moves to the successor vertex, from where the next round starts. The outcome 
is an infinite sequence of vertices, which is winning for the system if it satisfies 
the given objective. The associated quantitative satisfaction problem is to decide, 
given a threshold p, whether the system can win with probability at least p. The 
almost-sure problem is the special case where p = 1, and the positive problem is 
to decide whether the system can win with positive probability. The almost-sure 
and the positive problems are referred to as the qualitative satisfaction problems. 
When the answer to these decision problems is Yes, it is useful to construct a 
winning strategy for the system that can be used as a model for an implementation 
that ensures the objective be satisfied with the given probability. 

Traditional objectives in stochastic games are w-regular such as reachability, 
safety, and parity objectives [11], or quantitative such as mean-payoff objec- 
tives [16, 27]. For example, a parity objective may specify that every request of 
the environment is eventually granted by the system, and a mean-payoff objective 
may specify the long-run average power consumption of the system. A well- 
known drawback of parity and mean-payoff objectives is that only the long-run 
behaviour of the system may be specified [1, 9,21], allowing weird transient be- 
haviour: for example, a system may grant all its requests but with an unbounded 
response time; or a system with long-run average power consumption below 
some threshold may exhibit arbitrarily long (but finite) sequences with average 
power consumption above the threshold. This limitation has led to considering 
finitary versions of those objectives [9, 23, 8], where the sequences of undesired 
transient behaviours must be of fixed or bounded length. Window mean-payoff 
objectives [8] are quantitative finitary objectives that strengthen the traditional 
mean-payoff objective: the satisfaction of a window mean-payoff objective implies 
the satisfaction of the standard mean-payoff objective. Given a length £ > 1, the 
fixed window mean-payoff objective (FWMP(?)) is satisfied if except for a finite 
prefix, from every point in the play, there exists a window of length at most £ 
starting from that point such that the mean payoff of the window is at least a 
given threshold. In the bounded window mean-payoff objective (BWMP), it is 
sufficient that there exists some length £ for which the FWMP(¢) objective is 
satisfied. 


Contributions. We present algorithmic solutions for stochastic games with 
window mean-payoff objectives, and show that the positive and almost-sure 
problems are solvable in polynomial time for FWMP(£) (Theorem 5), and are in 
NP N coNP for BWMP (Theorem 6). The complexity result for the almost-sure 
problem entails that the quantitative satisfaction problem is in NPNcoNP (for both 
the fixed and bounded version), using standard techniques for solving stochastic 
games with prefix-independent objectives [13]. Note that the NP McoNP bound 
for the quantitative satisfaction problem matches the special case of reachability 
objectives in simple stochastic games [14], and thus would require a major 
breakthrough to be improved. 

As a consequence, using the FWMP(£) objective instead of the standard mean- 
payoff objective provides a stronger guarantee on the system, and even with a 
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polynomial complexity for the positive and the almost-sure problems (which is 
not known for mean-payoff objectives), and at no additional computational cost 
for the quantitative satisfaction problem. The solution relies on a reduction to 
non-stochastic two-player games (stochastic games without probabilistic vertices). 
The key result is to show that in order to win positively from some vertex of the 
game graph, it is necessary to win from some vertex of the non-stochastic game 
obtained by transforming all probabilistic vertices into adversarial vertices. While 
this condition, that we call the sure-almost-sure (SAS) property (Definition 1), 
was used to solve finitary Streett objectives [13], we follow a similar approach and 
generalize it to arbitrary prefix-independent objectives (Theorem 4). The bounds 
on the memory requirement of optimal strategies of Player 1 can also be derived 
from the key result, and are the same as optimal bounds for non-stochastic games. 
For the FWMP(¢£) and BWMP objectives in particular, we show that the memory 
requirement of Player 2 is also no more than the optimal memory required by 
winning strategies in non-stochastic games. 


As solving a stochastic game with a prefix-independent objective y reduces 
to solving non-stochastic games with objective y and showing that ọ satisfies 
the SAS property, we show that the FWMP(¢) and BWMP objectives satisfy the 
SAS property (Lemma 4, Lemma 5) and rely on the solution of non-stochastic 
games with these objectives [8] to complete the reduction. 


We improve the memory bounds for optimal strategies of both players in 
non-stochastic games. It is stated in [8] that |V| - £ memory suffices for both 
players, where |V| and @ are the number of vertices and the window length 
respectively. In [6, Theorem 2] and [19, Theorem 6.4], the bound is loosened 
to O(Wmax : £7) and O(wmax È -|V|) for Player 1 and Player 2 respectively, 
where Wmax is the maximum absolute payoff in the graph, as the original tighter 
bounds [8] were stated without proof. Since the payoffs are given in binary, this 
is exponential in the size of the input. In contrast, we tighten the bounds stated 
in [8]. We show that for Player 1, memory £ suffices (Theorem 1), and improve 
the bound on memory of Player 2 strategies as follows. We compute the set W of 
vertices from which Player 2 can ensure that the mean payoff remains negative 
for Z steps, as well as the vertices from which Player 2 can ensure that the game 
reaches W. These vertices are identified recursively on successive subgames of the 
original input game. If k is the number of recursive calls, then we show that k - £ 
memory suffices for Player 2 to play optimally (Theorem 2). Note that k < |V]. 
We also provide a lower bound on the memory size for Player 2. Given £ > 2, for 
every k > 1, we construct a graph with a set V of vertices such that Player 2 
requires at least k + 1 = $(|V| — £+ 3) memory to play optimally (Theorem 3). 
This is an improvement over the result in [8] which showed that memoryless 
strategies do not suffice for Player 2. Our result is quite counterintuitive since 
given an open window (a window in which every prefix has a total payoff less 
than 0) that needs to be kept open for another j < £ steps from a vertex v, one 
would conjecture that it is sufficient for a Player 2 winning strategy to choose an 
edge from v that leads to the minimum payoff over paths of length j. Thus for 
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every j, Player 2 should choose a fixed edge and hence memory of size £ should 
suffice. However, we show that this is not the case. 

To the best of our knowledge, this work leads to the first study of stochastic 
games with finitary quantitative objectives. 


Related work. Window mean-payoff objectives were first introduced in [8] for 
non-stochastic games, where solving FWMP(£) was shown to be in PTIME and 
BWMP in NPfMcoNP. These have also been studied for Markov decision processes 
(MDPs) in [4,3]. In [4], a threshold probability problem has been studied, while 
in [3], the authors studied the problem of maximising the expected value of 
the window mean-payoff. Positive, almost-sure, and quantitative satisfaction of 
BWMP in MDPs are in NP N coNP [4], the same as in non-stochastic games. 

Parity objectives can be viewed as a special case of mean-payoff objectives [22]. 
A bounded window parity objective has been studied in [9, 20, 12] where a play 
satisfies the objective if from some point on, there exists a bound £ such that from 
every state with an odd priority, a smaller even priority occurs within at most £ 
steps. Non-stochastic games with bounded window parity objectives can be solved 
in PTIME [20, 12]. Stochastic games with bounded window parity objectives have 
been studied in [13] where the positive and almost-sure problems are in PTIME 
while the quantitative satisfaction problem is in NP McoNP. A fixed version of 
the window parity objective has been studied for two-player games and shown 
to be PSPACE-complete [26]. Another window parity objective has been studied 
in [5] for which both the fixed and the bounded variants have been shown to be 
in PTIME for non-stochastic games. The threshold problem for this objective has 
also been studied in the context of MDPs, and both fixed and bounded variants 
are in PTIME [4]. Finally, synthesis for bounded eventuality properties in LTL is 
2-EXPTIME-complete [23]. 

Due to lack of space, some of the proofs have been omitted. A full version of 
the paper can be found in [15]. 


2 Preliminaries 


Stochastic games. We consider two-player turn-based zero-sum stochastic 
games (or simply, stochastic games in the sequel). The two players are referred 
to as Player 1 and Player 2. A stochastic game is a weighted directed graph 
G = ((V, E), (Vi, V2, Vo), P, w), where: 


— (V, E) is a directed graph with a finite set V of vertices and a set E CV x V 
of directed edges such that for all vertices v € V, the set E(v) = {v € V | 
(v, v’) € E} of out-neighbours of v is nonempty, i.e., E(v) Æ Ø (no deadlocks); 

— (Vi, V2, Vo) is a partition of V. The vertices in V, belong to Player 1, the 
vertices in V2 belong to Player 2, and the vertices in Vo are called probabilistic 
vertices (in figures, Player 1 vertices are shown as circles, Player 2 vertices as 
boxes, and probabilistic vertices as diamonds, and we use pronouns “she/her” 
for Player 1 and “he/him” for Player 2); 
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— P: Vo > D(V), where D(V) is the set of probability distributions over 
V, is a transition function that maps probabilistic vertices v € Vo to a 
probability distribution P(v) over the set E(v) of out-neighbours of v such 
that P(v)(v’) > 0 for all v’ € E(v) (i.e., all out-neighbours have nonzero 
probability); for the algorithmic and complexity results, we assume that 
probabilities are given as rational numbers. 

— w: E —> Q is a payoff function assigning a rational payoff to every edge in 
the game. 


Stochastic games are played in rounds. The game starts by initially placing 
a token on some vertex. At the beginning of a round, if the token is on a 
vertex v, and v € V; for i € {1,2}, then Player i chooses an out-neighbour 
v’ € E(v); otherwise v € Vo, and an out-neighbour vu’ € E(v) is chosen with 
probability P(v)(v’). Player 1 receives from Player 2 the amount w(v, v’) given by 
the payoff function, and the token moves to v’ for the next round. This continues 
ad infinitum, resulting in an infinite sequence 7 = ugvjv2--- E€ V“ such that 
(vi, vi+1) E€ E for all i > 0, called a play. For i < j, we denote by z(i,7) the 
infix vivii: vj of m. Its length is |r(i,7)| = j — i, the number of edges. We 
denote by 7(0,7) the finite prefix vovi -vj of m, and by a(t, 00) the infinite 
suffix vjvj41... of m. We denote by Playsg and Prefsg the set of all plays and 
the set of all prefixes in G respectively; the symbol G is omitted when it can 
easily be derived from the context. We denote by First(p) and Last(p) the first 
vertex and the last vertex of a prefix p € Prefsg respectively. We denote by Prefs 
(i € {1,2}) the set of all prefixes p such that Last(p) € V;. 


Objectives. An objective y is a Borel-measurable subset of Playsg [2]. A play 
m € Playsg satisfies an objective y if 7 € p. In a (zero-sum) stochastic game G 
with objective y, the objective of Player 1 is y, and the objective of Player 2 
is the complement set @ = Playsg \ y. Common examples of objectives are 
qualitative objectives such as reachability, safety, Biichi, and coBiichi. 

An objective y is closed under suffixes if for all plays 7 satisfying y, all 
suffixes of m also satisfy y, that is, m(j, o0) € y for all j > 0. An objective ọ is 
closed under prefixes if for all plays m satisfying y, for all prefixes p such that the 
concatenation p- 7 is a play in G, i.e., p-m € Playsg, we have that p-m € p. An 
objective y is prefiz-independent if it is closed under both prefixes and suffixes. 
An objective ọ is closed under suffixes if and only if the complement objective Y 
is closed under prefixes. Thus, an objective y is prefix-independent if and only if 
its complement @ is prefix-independent. 


Strategies. A (deterministic) strategy for Player i € {1,2} in a game G isa 
function c; : Prefs% — V that maps prefixes ending in a vertex v € V; toa 
successor of v. The set of all strategies of Player i € {1,2} in the game G is 
denoted by A;. Strategies can be realised as the output of a (possibly infinite- 
state) Mealy machine. A Mealy machine is a deterministic transition system with 
transitions labelled by an input/output pair. Formally, a Mealy machine M is 
a tuple (Q, qo, Xi, Xo, A, 6) where Q is the set of states of M (the memory of 
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the induced strategy), qo € Q is the initial state, X; is the input alphabet, X 
is the output alphabet, A: Q x Xi > Q is a transition function that reads the 
current state of M and an input letter and returns the next state of M, and 
ô: Q x Xi > Xo is an output function that reads the current state of M and 
an input letter and returns an output letter. We point the reader to [15] for a 
description of how a strategy is defined by a Mealy machine. 

The memory size of a strategy g; is the smallest number of states a Mealy 
machine defining c; can have. A strategy c; is memoryless if oi(p) only depends 
on the last element of the prefix p, that is for all prefixes p,p’ € Prefsg if 
Last(p) = Last(p’), then o;(p) = o;(p’). Memoryless strategies can be defined by 
Mealy machines with only one state. 

A play m = wpv1--- is consistent with a strategy o; E€ A; (i € {1,2}) if 
vj+1 = oi(7(0,7)) for all j > 0 such that v; € V;. A play m is an outcome of 
cı and ov if it is consistent with both c1 and o2. We denote by Pigs. (y) the 
probability that an outcome of gı and o2 in G with initial vertex v satisfies y. 


Non-stochastic two-player games. A stochastic game without probabilistic 
vertices (that is, with Vo = Ø) is called a non-stochastic two-player game (or 
simply, non-stochastic game in the sequel). In a non-stochastic game G with 
objective y, a strategy g; is winning from a vertex v € V for Player i (i € {1, 2}) 
if every play in G with initial vertex v that is consistent with g; satisfies the 
objective y. A vertex v € V is winning for Player i in G if Player i has a winning 
strategy in G from v. The set of vertices in V that are winning for Player i in G 
is the winning region of Player i in G, denoted ((i))¢(y). If a vertex v belongs to 
the winning region of Player i (i € {1,2}), then Player i is said to play optimally 
from v if she follows a winning strategy. 


Subgames. Given a stochastic game G = ((V, E), (Vi, V2, Vo), P, w), a subset 
V’ C V of vertices induces a subgame if (i) every vertex v’ € V’ has an out- 
neighbour in V’, that is E(v') NV’ Æ Ø, and (ii) every probabilistic vertex 
v € Vo OV’ has all out-neighbours in V’, that is E(v’) C V’. The induced 
subgame is ((V’, E’), (VLO V’, VAV’, Vo NV’), P, w), where FE’ = EN(V'’xV’), 
and P’ and w’ are restrictions of P and w respectively to (V’, E’). We denote 
this subgame by G [ V’. Let y be an objective in the stochastic game G. We 
define the restriction of y to a subgame G’ of G to be the set of all plays in G’ 
satisfying y, that is, the set Playsg,N y. 


Satisfaction probability. A strategy gı of Player 1 is winning with probability p 
from an initial vertex v in G for objective y if Pro” (p) = p for all strategies 
o2 of Player 2. A strategy cı of Player 1 is positive winning (resp., almost-sure 
winning) from v for Player 1 in G with objective ọ if Prg? (p) > 0 (resp., 
Pro’ (p) = 1) for all strategies o2 of Player 2. We refer to positive and almost- 
sure winning as qualitative satisfaction of p, while for arbitrary p € [0, 1], we call 
it quantitative satisfaction. We denote by ((1))6°(~) (resp., by (1))45(~)) the 
positive (resp., almost-sure) winning region of Player 1, i.e., the set of all vertices 


in G from which Player 1 has a positive (resp., almost-sure) winning strategy 
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for G with objective vy. If a vertex v belongs to the positive (resp., almost-sure) 
winning region of Player 1, then Player 1 is said to play optimally from v if 
she follows a positive (resp., almost-sure) winning strategy from v. We omit 
analogous definitions for Player 2. 


Positive attractors and traps. The Player i positive attractor (i € {1,2}) to 
T CV, denoted PosAttr;(T), is the set of vertices in V from which Player 7 can 
ensure that the token reaches a vertex in T with positive probability. It is possible 
to compute the positive attractor in O(|E|) time [10]. In non-stochastic games, a 
positive attractor to a set T is the same as an attractor to the set T, which we 
denote by Attr;(T). Computation of PosAttr;(T) gives a memoryless strategy for 
Player i that ensures that the token reaches T with positive probability. We call 
such a strategy a positive-attractor strategy of Player i. 

A trap for Player 1 is a set T C V such that for every vertex v € T, if 
v E€ Vi U Vg, then E(v) C T, and if v € V2, then E(v) NT Æ Ø. In other words, 
from every vertex v € T, Player 2 can ensure (with probability 1) that the token 
never leaves T, moreover using a memoryless strategy. A trap for Player 2 can 
be defined analogously. 


Remark 1. Let G be a non-stochastic game with objective y for Player 1. If y is 
closed under suffixes, then the winning region of Player 1 is a trap for Player 2. 
As a corollary, if y is prefix-independent, then the winning region of Player 1 is 
a trap for Player 2 and the winning region of Player 2 is a trap for Player 1. 


3 Window mean payoff 


We consider two types of window mean-payoff objectives, introduced in [8]: (i) 
fixed window mean-payoff objective (FWMP(é)) in which a window length £ > 1 
is given, and (ii) bounded window mean-payoff objective (BWMP) in which for 
every play, we need a bound on window lengths. We define these objectives below. 

For a play z in a stochastic game G, the total payoff of an infix r(i,i +n) = 
UsVit1'+*Vitn is defined as TP(7(i,i+n)) = ETTI w(u, V41). The mean 
payoff of an infix m(i,i + n) is defined as MP(r(i,i + n)) = +TP(r(i,i + n)). 
Observe that the mean payoff of an infix is nonnegative if and only if the total 
payoff of the infix is nonnegative. The mean payoff of a play m is defined as 
MP(T) = lim inf MP((0, 7)). Given a window length £ > 1, a play 7 = vov ++: 


in G satisfies the fixed window mean-payoff objective FWMPg(é) if from every 
position after some point, it is possible to start an infix of length at most £ with 
a nonnegative mean payoff. Formally, 


FWMPo¢(¢) = {r € Playsg | Ik > 0-Wi > k-3j € {1,..., 0} : MP(x(é,i+j)) > 0}. 


We omit the subscript G when it is clear from the context. Note that when £ = 1, 
the FWMP(1) and FWMP(1) (i.e., the complement of FWMP(1)) objectives 
reduce to coBiichi and Büchi objectives respectively. The following properties 
of FWMP(£) have been observed in [8]. For all window lengths £ > 1, if a play 
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m satisfies FWMP (£), then MP(7) > 0. In all plays satisfying FWMP (£), there 
exists a suffix that can be decomposed into infixes of length at most £, each with 
a nonnegative mean payoff. Such a desirable robust property is not guaranteed 
by the classical mean-payoff objective, where infixes of unbounded lengths may 
have negative mean payoff. 

As defined in [8], given a play 7 = ugv,--- and 0 < i < j, we say that the 
window 7(i, j) is open if the total-payoff of q(i, k) is negative for all i < k < j. 
Otherwise, the window is closed. Given j > 0, we say a window is open at j 
if there exists an open window z(i,j) for some i < j. The window starting 
at position i closes at position j if j is the first position after i such that the 
total-payoff of m(7, 7) is nonnegative. If the window starting at 7 closes at j, then 
for alli < k < j, the windows 7(k,7) are closed. This property is called the 
inductive property of windows. 

We also have the bounded window mean-payoff objective BWMP. A play a 
satisfies the BWMP objective if there exists a window length ¢ > 1 for which 7 
satisfies FWMP(£), i.e., 


BWMPg = {r € Playsg | £ > 1 : 7 € FWMP(8)} 


Equivalently, a play m does not satisfy BWMP if for every suffix of 7, for all £ > 1, 
the suffix contains an open window of length £. Note that both FWMP(¢) for all 
£ > 1 and BWMP are prefix-independent objectives. 


Decision problems. Given a game G, an initial vertex v € V, a rational 
threshold p € [0,1], and an objective y (that is either FWMP(é) for a given 
window length £ > 1, or BWMP), consider the problem of deciding: 


— Positive satisfaction of p: whether Player 1 positively wins ọ from v, i.e., 
whether v € ((1))§°5(y). 

— Almost-sure satisfaction of p: whether Player 1 almost-surely wins y from v, 
i.e., whether v € ((1))39(y). 

— Quantitative satisfaction of p (also known as quantitative value problem [13]): 
whether Player 1 wins y from v with probability at least p, i.e., whether 
SUP €11 infos€42 Pron” (p) >p. 


Note that these three problems coincide for non-stochastic games. As considered 
in previous works [8, 3, 4], the window length £ is usually small (typically £ < |V]), 
and therefore we assume that £ is given in unary (while the payoff on the edges 
is given in binary). From determinacy of Blackwell games [24], stochastic games 
with window mean-payoff objectives as defined above are determined, i.e., the 
largest probability with which Player 1 is winning and the largest probability 
with which Player 2 is winning add up to 1. 


Algorithms for non-stochastic window mean-payoff games. To compute 
the positive and almost-sure winning regions for Player 1 for FWMP(¢), we recall 
intermediate objectives defined in [8]. The good window objective GWg (£) consists 
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Algorithm 1 NonStocFWMP(G, £) Algorithm 2 NonStocDirFWMP(G, £) 


In: G = ((V, E), (Vi, V2,2),w) and £> 1 In: G =((V,E),(Wi,V2,2),w) and £ > 1 
Out: ((1))g(FWMP(£)) Out: ((1))¢(DirFWMP(2)) 


A & Attrı (Wa) 
return AUNonStocFWMP(G [ (V\ 
A), &) 


1: Wa < NonStocDirFWMP(G, £) 1: Wow + GoodWin(G, £) 

2: if Wa = Ø then 2: if Wow = V or Wow = Ø then 
3: return @ 3: | return Wgw 

4: else 4: else 

5: 5: 

6: 6: 


return NonStocDirFWMP(G [ 


A & Attro(V \ Wow) 
(Wow \ A), 8. 


of all plays m in G such that the window opened at the first position in the play 
closes in at most @ steps: 


GWe() = {r € Playsg | 3j € {1,..., 2} : MP(x(0,7)) > 0} 


The direct fixed window mean-payoff objective DirFWMPg(é) consists of all 
plays m in G such that from every position in 7, the window closes in at most £ 
steps: 

DirFWMPg (£) = {r € Playsg | Vi > 0: 1(i,00) € GWe(é)} 


The FWMPg(£) objective can be expressed in terms of DirFWMPg(£): 


FWMPo¢(£) = {r € Playsg | Ik > 0: (hk, 00) € DirFWMPog(£)} 


We refer to Algorithms 1, 2, and 3 from [8] shown here with the same numbering. 
They compute the winning regions for Player 1 for the FWMP(£), DirF WMP (£), 
and GW(é) objectives in non-stochastic games respectively. The original algo- 
rithms in [8] contain subtle errors for which the fixes are known [6,19]. For 
completeness, we refer the reader to [15] for counterexamples for the algorithms 
in [8] along with brief explanations of correctness for the modified versions. 

Algorithm 3 uses dynamic programming to compute, for all v € V and all 
lengths 7 € {1,..., 4}, the largest payoff C;(v) that Player 1 can ensure from v 
within at most i steps. The winning region for GW(£) for Player 1 consists of all 
vertices v such that Ce(v) > 0. 


4 Memory requirement for non-stochastic window 
mean-payoff games 


The memory requirement for winning strategies of Player 1 in non-stochastic 
games with objective FWMP(Z£) is claimed to be O(|V|- £) without proof [8, 
Lemma 7], and further “correctly stated” as O(wmax : Ê), where Wmax is the 
maximum absolute payoff in the graph [6, Theorem 2]. We improve upon these 
bounds and show that memory of size @ suffices for a winning strategy of Player 1. 
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Algorithm 3 GoodWin(G, £) 


In: G = ((V, E), (Wi, V2, 2), w) the non-stochastic game, and £ > 1, the window length 
Out: The set of vertices from which Player 1 wins GW(£) in G 

1: for all v € V do 
Co(v) -0 
for all i € {1,...,4} do 
L Ci(v) + —oo 
for alli € {1,...,2} do 
for all v € Vi do 
, Ci(v) © maxiy,oyee{max{w(v, v’), w(v,v’) + Ci_-i(v')}} 
or all v € Vo do 

9: | Ci(v) & minw wjer{max{w(v, v’), w(v, v) + Ci_1(v')}} 
10: Wow & {v € V | Ce(w) > 0} 
11: return Wow 


We also present a family of games with arbitrarily many vertices where Player 2 is 
winning and all his winning strategies require at least $(|V|—@) +3 memory, while 
it was only known that memoryless strategies are not sufficient for Player 2 [8]. 


4.1 Memory requirement for Player 1 for FWMP objective 


Upper bound on memory requirement for Player 1. We show that memory 
of size @ suffices for winning strategies of Player 1 for the DirFWMP(£) objective 
(Lemma 1), which in turn shows that the same memory also works for the 
FWMP (£) objective (Theorem 1). 


Lemma 1. /f Player 1 wins in a non-stochastic game with objective DirFWMP(é), 
then Player 1 has a winning strategy with memory of size £. 


Proof (Sketch). Given a non-stochastic game G, let Wa be the winning region 
of Player 1 in G for objective DirFWMP(£). By definition, every vertex in Wa is 
also winning for Player 1 for the GW(£) objective. 

A winning strategy og of Player 1 in Wa satisfies the objective GW(£) by 
closing a window within at most £ steps and then restarts with the same strategy, 
playing for GW(@) and so on. Using memory space Q = {1,..., 2}, we may store 
the number of steps remaining before the window must close. However, the 
window may close any time within £ steps, and the difficulty lies in detecting this 
independently of the history. For memory state q = į and the next visited vertex 
being v, intuitively, the memory should be updated to q =i — 1 if the window 
did not close yet upon reaching v, and to q = £ if it did, but that depends on 
which path was followed to reach v (not just on v), which is not stored in the 
memory space. 

The crux is to show that it is not always necessary for Player 1 to be able 
to infer when the window closes. Given the current memory state q = i, and 
the next visited vertex v, the memory update is as follows: if C;(v) > 0 (that 
is, Player 1 can ensure the window from v will close within i steps), then we 
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update to q = i — 1 (decrement) although the window may or may not have 
closed upon reaching v; otherwise C;(v) < 0 and we update to q = ¢— 1 (reset to 
L and decrement) and we show that in this case the window did close. Intuitively, 
updating to q = i — 1 is safe even if the window did close, because the strategy 
of Player 1 will anyway ensure the (upcoming) window is closed within i—1 < £ 
steps. A formal description of a Mealy machine with £ states defining a winning 
strategy of Player 1 for the DirFWMP(2) objective is given in [15]. 


Theorem 1. If Player 1 wins in a non-stochastic game G with objective FWMP (£), 
then Player 1 has a winning strategy with memory of size £. 


Proof (Sketch). Since FWMP(¢) is a prefix-independent objective, we have that 
the winning region ((1))g(FWMP(¢)) of Player 1 is a trap for Player 2 (Re- 
mark 1), and induces a subgame, say Go. Let there be k + 1 calls to the 
subroutine NonStocDirFWMP from Algorithm 1 where k < |V|. We denote 
by (Wi)ieq.,...,n} the nonempty W4 returned by the itt call to the subroutine, 
and let A; = Attr,;(W;). The A,’s are pairwise disjoint, and their union is 
Dan A; = (1))g(FWMP(é)). For i € {1,...,k}, inductively define G; to be the 
subgame induced by the complement of A; in G;—1. Since DirFWMP(£) is closed 
under suffixes, for all i € {1,...,4}, we have that W; is a trap for Player 2 in G; 
(Remark 1). 

We construct a strategy oS that follows the (memoryless) attractor strategy 
in (U; (A; \ Wi), and follows the winning strategy oq for DirFWMP(é) objective 
(defined in the proof of Lemma 1) in U; W;. The reader is pointed to [15] for 
a formal description of a Mealy machine defining the strategy oS. For the 
correctness of the construction, the crux is to show that one of the sets W; 
(i€ {1,...,k}) is never left from some point on. Intuitively, given the token is 
in A; for some i € {1,...,k} (thus in G;), following oS, the token will either 
remain in Aj, or leave the subgame G; and enter A; for a smaller index j < i. 
The result follows since this can be done at most k times. 


Lower bound on memory requirement for Player 1. In [8], the authors 
show a game with £ = 4 where Player 1 requires memory at least 3. This can be 
generalized to arbitrary £ to show that memory of size £ — 1 may be necessary 
(See [15] for details). 


4.2 Memory requirement for Player 2 for FWMP objective 


Upper bound on memory requirement for Player 2. Now we show that 


for the FWMP(¢) objective, Player 2 has a winning strategy that uses memory 
of size at most |V|- @. This has been loosely stated in [8] without a formal proof. 


Theorem 2. Let G be a non-stochastic game with objective FWMP (£) for Player 2. 
Then, Player 2 has a winning strategy with memory size at most |V| - £. 


Proof (Sketch). Since FWMP(8) is a prefix-independent objective, so is FWMP (£). 


We have that ((2))¢(FWMP(¢)) is a trap for Player 1 (Remark 1) and induces a 
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subgame, say Ho, of G. Let there be k + 1 calls to the subroutine GoodWin from 
Algorithm 2 (where k < |V|), and let H; be the subgame corresponding to the i‘ 
call of the subroutine. We denote by (W;)ie41,...,n} the complement of Wy in Hi, 
where Wy» is returned by the it? call to the subroutine, and let A; = Attro(W;). 
The A;’s are pairwise disjoint, and their union is ble A; = ((2))g(FWM P(é)). 
We describe a winning strategy for the FWMP(£) objective with memory 
k - £, which is at most |V|- £. The strategy is always in either attractor mode 
or window-open mode. When the game begins, it is in attractor mode. If the 
strategy is in attractor mode and the token is on a vertex v € A; \ W; for some 
i € {1,...,k}, then the attractor strategy is to eventually reach W;. If the token 
reaches W;, then the strategy switches to window-open mode. Since all vertices 
in W; are winning for Player 2 for the GW(¢) objective, he can keep the window 
open for £ more steps, provided that Player 1 does not move the token out of 
the subgame H;. If, at some point, Player 1 moves the token out of the subgame 
Hi to A; for a smaller index j < i, then the strategy switches back to attractor 
mode, this time trying to reach W; in the bigger subgame Hj. Otherwise, if 
Player 2 keeps the window open for £ steps, then the strategy switches back to 
attractor mode until the token reaches a vertex in UŁ: Wi. This strategy can 
be defined by a Mealy machine MNS with states {1,...,k} x {1,..., 2}, where 
the first component tracks the smallest subgame H; in which the window started 
to remain open, and the second component indicates how many more steps the 
window needs to be kept open for. A formal description of MNS can be found 
in [15]. 


Lower bound on memory requirement for Player 2. In [8], it was shown 
that memoryless strategies do not suffice for Player 2. We improve upon this 
lower bound. Given a window length £ > 2, for every k > 1, we construct a game 
Gre with 2k + £ — 1 vertices such that every winning strategy of Player 2 in Gk, 
requires at least k + 1 memory. 


Theorem 3. There exists a family of non-stochastic games {Grehe>1,¢>2 with 
objective FWMP(é) for Player 1 and edge weights in {—1,0,+1} such that every 
winning strategy of Player 2 requires at least (|V| —£+1)+1 memory, where 
|V| = 2k +@-1. 


Proof (Sketch). Let A = {a1,...,ax}, B = {bi,..., bk}, and C = {c1,...,ce_-i} 
be pairwise disjoint sets. The vertices of Gg are AU BUC with Vj = AUC and 
V2 = B. Figure 1 shows the game G43. A more formal description of Gg, can be 
found in [15]. 

Observe that the only open windows of length £ in the game G;, ¢ are sequences 
of the form apbrCe—1 +++ cı for all p < r. Also note that Player 2 has a winning 
strategy that wins starting from every vertex in the game, as Player 2 can force 
the token to eventually take a red edge followed by two black edges. 

When the token reaches a vertex b, € B, Player 2 can either move the token 
to a, € A or to ce_; E€ C. Depending on which vertex the token was on before 
reaching b,, one of the two choices is good for Player 2. If the token reaches b, 
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Figure 1: Game Ga,3 with parameter k = 4 and window length £ = 3. Red edges (from 
ap to br for p < r) have payoff —1, black edges (from b, to c2) have payoff 0, and blue 
edges (the remaining edges) have payoff +1. 


Table 1: Good choices x(u, br) for all u € AU {ci} and br € B in the game Ga,3 
aiby > ce az2bı =F Qi 
aıb2 > ce az2b2 > C2 a3b2 —> a2 
aıb3 —> C2 a2b3 > c2 a3b3 — C2 a4b3 > a3 
aiba =} ù az2b4 =} Cə a3b4 =} C2 aaba —> C2 Ciba — a4 


from a, for p < r, then it is good for Player 2 to move the token to ce_; € C so 
that the window starting at ap remains open for £ steps. Otherwise, if the token 
reaches b, from a,41, then it is good for Player 2 to move the token to ar so that 
an edge with negative payoff may eventually be taken. For all u € AU {c1}, for 
all b, € B such that (wu, br) is an edge in Gk e, we denote by y(u, b,) the vertex 
ar or ce—1 that is good for Player 2. We list the good choices in the game G4,3 in 
Table 1. The columns are indexed by u € AU {cı} and the rows are indexed by 
b, € B. 

We show that for each column in the table, there exists a distinct memory 
state in every Mealy machine defining a winning strategy of Player 2. This gives 
a lower bound of k + 1 on the number of states of such a Mealy machine. Since 
Gp, has 2k + £—1 vertices, the memory requirement of a winning strategy of 
Player 2 is at least ¿(|V| — 4+1) +1. 


Given a winning strategy oS of Player 2 for the FWMP(é) objective, the 
following lemma gives an upper bound on the number of steps between consecutive 
open windows of length £ in any play consistent with o!!S. This lemma is used in 
Section 6, where we construct an almost-sure winning strategy of Player 2 for 


the FWMP(£) objective. 


Lemma 2. Let G be a non-stochastic game such that ((2))g(FWMP(¢)) = V. Let 
NS b % _ ` + . + 
a” be a finite-memory strategy of Player 2 of memory size M that is winning 


for FWMP(£) from all vertices in G. Then, for every play m of G consistent with 


oNS, every infix of m of length M - |V| - £ contains an open window of length £. 


The proof is based on the pigeonhole principle and appears in [15]. 
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Figure 2: Büchi objective does not satisfy the SAS property in this game. 


5 Reducing stochastic games to non-stochastic games 


For a stochastic game G, let Gus = ((V, E), (Vi, V2U V, Ø), w) be the (adversarial) 
non-stochastic game corresponding to G, obtained by changing all probabilistic 
vertices in G to Player 2 vertices. In [13], a property of finitary Streett objective 
was used to solve stochastic games by reducing them to non-stochastic games 
with the same objective. In this section, we generalize this property for arbitrary 
prefix-independent objectives. 


Definition 1 (Sure-almost-sure (SAS) property). A prefix-independent ob- 
jective p in a game G satisfies the SAS property if ((2))g,.(%) = V implies 
(aye (P) = V, that is, if Player 2 wins the objective Y from every vertex in Gus, 
then Player 2 almost-surely wins the same objective Y from every vertex in G. 


Every prefix-independent objective satisfies the converse of the SAS property 
since if Player 2 even wins positively from all vertices in G, then since he controls 
all probabilistic vertices in Gus, he wins from all vertices in Gus by choosing 
optimal successors of probabilistic vertices. We show in Section 6 that for all 
stochastic games G, the objectives FWMP(£) and BWMP satisfy the SAS property, 
while in Example 1, we show that there exists a stochastic game in which Büchi 
objective does not satisfy the SAS property. 


Example 1. Consider the game G in Figure 2. The objective vy in this game is 
a Buchi objective: a play m satisfies the Biichi objective if m visits vertex vı 
infinitely often. Although from every vertex, with positive probability (in fact, 
with probability 1), a play visits vı infinitely often, from none of the vertices, 
Player 1 can ensure the Büchi objective in the non-stochastic game Gns. 


Theorem 4 gives complexity bounds for solving stochastic games with ob- 
jectives satisfying the SAS property in terms of the complexity of solving non- 
stochastic games with the same objective. 


Theorem 4. Given G and vy, suppose in every subgame G’ of G, the objective 
restricted to G’ satisfies the SAS property. Let NonStocWin, (Gus) be an algorithm 
computing (1))g,.(~) in Gus in time C. Then, the positive and almost-sure 
satisfaction of p can be decided in time O(\V|-(C+|E])) and O(|V|? -(C+]|E})) 
respectively. 

Moreover, for positive and almost-sure satisfaction of p, the memory require- 
ment for Player 1 to play optimally in stochastic games is no more than that for 
non-stochastic games. 


48 L. Doyen et al. 


Algorithm 4 PosWin, (G) Algorithm 5 ASWin, (G) 

In: G = ((V, E), (Vi, V2, Vo), P,w) andy In: G = ((V, E), (Vi, V2, Vo), P, w) and y 
Out: (1)E(y) Out: ((1)38(¢) 

1: W, + NonStocWin, (Gus) 1: W2 + V \ PosWin, (G) 

2: if W, = Ø then 2: if W = Ø then 

3: | return © 3: | return V 

4: else 4: else 

5: A, © PosAttri(W,) 5: A> + PosAttre(W2) 

6: | return A UPosWino (G | (V\Ai)) 6: | return ASWinọ (G | (V \ 42)) 


Theorem 4 does not give bounds on the memory requirement for winning strategies 
of Player 2 for objective y in the stochastic game, but we provide such bounds 
specifically for FWMP (£) and BWMP in Section 6. We give a sketch of the proof 
of Theorem 4 below. The complete proof appears in [15]. 

The algorithms to compute the positive and almost-sure winning regions 
in G, and their proofs of correctness are the same as in the case of finitary 
Streett objectives described in [13]. The PosWin, algorithm (Algorithm 4) uses 
NonStocWin, as a subroutine to compute ((1))§°(y). The fact that y satisfies 
the SAS property is used to show the correctness of this algorithm. The depth 
of recursive calls of this algorithm is bounded above by |V|, which gives the 
complexity bound. The ASWing algorithm (Algorithm 5) in turn uses PosWing 
as a subroutine to compute the ((1))4°(~). The depth of recursive calls of this 
algorithm is also bounded above by |V|, which gives the complexity bound. The 
following lemma, which is a special case of Theorem 1 in [7], is used to show the 
correctness of this algorithm. 


Lemma 3. /7, Theorem 1] For a stochastic game G with prefiz-independent 
objective p, if ((2))G°(~) =V, then (2))¢°(¢) =V. 


For both positive and almost-sure winning, Player 1 does not require any 
additional memory in the stochastic game compared to the non-stochastic game. 
We describe a strategy of? of Player 1 that is positive winning from all vertices 
in ((1))6°°(v). In each recursive call to PosWin, algorithm, from every vertex in 
W,, the strategy ofS mimics a winning strategy of Player 1 in Gus, while for 
vertices in A, \ Wj, it follows a memoryless attractor strategy to reach W,. The 
same strategy is almost-sure winning for Player 1 from all vertices in ((1))4°(y). 

Finally, we look at the quantitative decision problem. The quantitative sat- 
isfaction for y can be decided in NP? ([13, Theorem 6]), where B is an oracle 
deciding positive and almost-sure satisfaction problems for y. It is not difficult 
to see that the quantitative satisfaction for y can be decided in NP? N coNP®. 
Moreover, from the proof of [13, Theorem 6], it follows that the memory require- 
ment of winning strategies for both players for the quantitative decision problem 


is no greater than that for the qualitative decision problem. 
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Corollary 1. Given G and p as described in Theorem 4, let B be an oracle 
deciding the qualitative satisfaction of p. Then, the quantitative satisfaction of p 
is in NP? N coNP®. Moreover, the memory requirement of optimal strategies for 
both players is no greater than that for the positive and almost-sure satisfaction 


of yp. 


6 Reducing stochastic window mean-payoff games: A 
special case 


In this section, we show that for all stochastic games G and for all £ > 1, the 
objectives FWMPg(l) and BWMPg, which are prefix-independent, satisfy the 
SAS property of Definition 1. Thus, by Theorem 4, we obtain bounds on the 
complexity and memory requirements of Player 1 for the positive and almost-sure 
satisfaction of these objectives. We also show that for both these objectives, the 
memory requirements of Player 2 to play optimally for positive and almost-sure 
winning in stochastic games is no more than that of the non-stochastic games. The 
algorithms to compute the positive and almost-sure winning regions of Player 1 
for both FWMP(é£) and BWMP objectives are obtained by instantiating y equal 
to FWMP(é) and BWMP in Algorithms 4 and 5. Thus, we obtain the algorithms 
PosWinewmp(e); ASWingwmp(e); PosWingwwp, and ASWingwwp.- 


6.1 Fixed window mean-payoff objective 
We first discuss the SAS property for the FWMP (£) objective. 


Lemma 4. In stochastic games, for all £>1, the FWMP(£) objective satisfies 
the SAS property. 


Proof (Sketch). We show that for all stochastic games G, if ((2)) gus (FWMP (2) = 
V, then ((2))8S(FWMP(2)) = V. If (2))g,.(FWMP(¢)) = V, then from Theorem 2, 
there exists a finite-memory strategy oS (say, with memory M) of Player 2 
that is winning for objective FWMP(é) from every vertex in Gus. Given such a 
strategy, we construct below a strategy o^ of Player 2 in the stochastic game G 
that is almost-sure winning for FWMP (£) from every vertex in G. 

In Gus, Player 2 controls vertices in V2 U Vo, while in G, Player 2 only controls 
vertices in V2 and the probability function P determines the successors of vertices 
in Vo. While the strategy o)!S is winning for FWMP(¢) from all vertices in Gus, 
it may not be almost-sure winning for FWMP (£) in G. This is because each time 
the token is on a probabilistic vertex, a deviation occurs with positive probability, 
i.e., the successor chosen by the distribution is not consistent with oS, resulting 
in a potentially worse outcome for Player 2. For example, in Figure 3, we see 
a stochastic game G and a Mealy machine MYS defining a strategy ofS that 
is winning for Player 2 from all vertices in the non-stochastic game Gy. In all 
outcomes in Gus that are consistent with on, the token never moves from ve 
to v7. However, in G, a deviation may lead the token to move along (vg, v7). This 
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| | 


va/v6, U5 /V6 


v6/vs, v7/vs 


v2/e, v3 /€ 


va/V6, U5/V6 
>i 


va/v6, vs /V6 v6/€, U7 /Us 


Figure 3: (top) Stochastic game G with objective FWMP(3) for Player 2. All unlabelled 
edges have payoff 0. (middle) Mealy machine MXS defining a strategy of that is 
winning from all vertices in Gus for FWMP(3). (bottom) Part of the Mealy machine 
M$ defining a reset strategy that is almost-sure winning from all vertices in G. 


results in a losing outcome for Player 2 as the token gets trapped in vg, and 
subsequently no window remains open for £ steps. Such harmful deviations can 
be detected, and starting with the strategy o!!S, we construct a strategy o4% that 
mimics gle as long as harmful deviations do not occur, and resets otherwise, i.e., 
the strategy forgets the prefix of the play before the deviation. For instance, when 
the token moves from vg to v7 in G, the strategy resets and the play continues as 
if the game began from v7. We call o}° a reset strategy. Figure 3 shows a part 
of a Mealy machine M?S defining a reset strategy for the game G. The figure 
contains all the reset transitions out of q4, but the reset transitions out of q1, 
q2, and q3 have been omitted for space. More details on how to obtain a Mealy 
machine that defines of5 from a Mealy machine that defines ofS without adding 
any new states can be found in [15]. 
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Now, we argue that the reset strategy is almost-sure winning for Player 2 from 
all vertices in G. If a play in G continues for M- |V|- £ steps without deviating, 
then by Lemma 2, it contains an open window of length @. From any point in 
the play, the probability that o}° successfully copies o}!S for i steps (that is, no 
deviations occur) is at least p’, where p is the minimum probability over all the 
edges in G. It follows that from every point in the play, the probability that an 
open window of length Z occurs in the next M-|V]|- £ steps is at least pM'IVI£, 
Therefore, from every position in the play, the probability that an open window of 
length £ occurs eventually is at least 57 j.9(1 — pV 12) . pMIVI£ = 1, Thus, with 
probability 1, infinitely many open windows of length @ occur in the outcome, and 
the outcome satisfies FWMP (£). Thus, all vertices in G are almost-sure winning 


for Player 2 for FWMP (£). For all stochastic games G, the objective FWMP (2) 
satisfies the SAS property. 


We now construct a strategy of of Player 2 that is positive winning from 


all vertices in ((2))§°S(FWMP(¢)). Let Wå and A} denote the sets W, and A, 
computed in the it? recursive call of the ASWinrwmap,e) algorithm respectively. If 
the token is in UJ; Wi, then o° mimics 45; if the token is in U; 44 \ Wå, then 
ofS is a positive-attractor strategy to W3 which is memoryless. Then, 05° is 
a positive winning strategy for Player 2 from all vertices in ((2))5°°(FWMP(2)). 


Using Theorem 4, Corollary 1, and Lemma 4, we have the following. 


Theorem 5. Given a stochastic game G, a window length £ > 1, and a threshold 
p € [0,1], for FWMP¢ (2), the positive and almost-sure satisfaction for Player 1 
are in PTIME, and the quantitative satisfaction is in NP McoNP. Moreover for 
optimal strategies, memory of size £ is sufficient for Player 1 and memory of size 
|V| -L is sufficient for Player 2. 


6.2 Bounded window mean-payoff objective 


We show that the SAS property holds for the BWMP objective for all stochastic 
games G. 


Lemma 5. In stochastic games, the BWMP objective satisfies the SAS property. 


Proof (Sketch). We show that for all stochastic games G, if ((2))¢..(BWMP) = V, 
then ((2))45(BWMP) = V. Since every play that satisfies BWMP also satisfies 
FWMP(¢) for all £ > 1, if (2))¢,.(BWMP) = V, then (2))¢,.(FWMP(¢£)) = V. It 


follows that for each £ > 1, Player 2 has a finite-memory strategy (say, with 
memory Me), that is winning for the FWMP(2) objective from all vertices in Gus. 
For every such strategy, we construct a reset strategy of of memory size at most 
My as described in the proof of Lemma 4 that is almost-sure winning for the 
FWMP(¢) objective from all vertices. We use these strategies to construct an 
infinite-memory strategy a$$ of Player 2 that is almost-sure winning for BWMP 
from all vertices in the stochastic game G. 

Let p be the minimum probability over all edges in the game, and for all £ > 1, 


let q(£) denote pMeIVI£, We partition a play of the game into phases 1,2,... such 
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that for all > 1, the length of phase £ is equal to Mọ - |V|- £- [1/q(€)]. We define 
the strategy oe? as follows: if the game is in phase £, then os is on the reset 
strategy that is almost-sure winning for FWMP(£) in G. 

We show that o4% is almost-sure winning for Player 2 for BWMP in G. Let Ee 
denote the event that phase £ contains an open window of length £. Given a play 7, 
if E; occurs in 7 for infinitely many £ > 1, then for every suffix of m and for all 
£ > 1, the suffix contains an open window of length £, and 7 satisfies BWMP. For 
all £ > 1, we compute the probability that E occurs in the outcome. For all £ > 1, 
we can divide phase £ into [1/q(@)] blocks of length Me-|V|-@ each. If at least one 
of these blocks contains an open window of length £Z, then the event Ez occurs. It 
follows from the proof of Lemma 4 that if Player 2 follows of, then the probability 
that there exists an open window of length @ in the next Me - |V|- Z steps is at 
least q(€). Hence, the probability that none of the blocks in the phase contains 
an open window of length Z is at most (1 — q(¢))!'/41. Thus, the probability 
that Ey occurs in phase £ is at least 1 — (1 — q(@))1/271 > 1 — 4 ~ 0.63 > 0. It 
follows that with probability 1, for infinitely many values of £ > 1, the event E, 
occurs in 7. 


Note that solving a non-stochastic game with the BWMP objective is in 
NP McoNP [8]. Thus by Corollary 1, quantitative satisfaction for BWMP is in 
NPNPONP A coNPNPOCNP which is the same as NP N coNP [25]. 

Moreover, from [8], Player 1 has a memoryless strategy and Player 2 needs 
infinite memory to play optimally in non-stochastic games with the BWMP 
objective. From the proof of Lemma 5, by using the strategy ofS, Player 2 
almost-surely wins BWMP from all vertices in ((2))4°(BWMP). We can construct 
a positive winning strategy of°* for Player 2 from all vertices in ((2))6°S(BWMP) 
in a similar manner as done for the positive winning strategy for FWMP(¢) in 
Section 6.1. We summarize the results in the following theorem: 


Theorem 6. Given a stochastic game G and a threshold p € [0,1], for BWMPg, 
the positive, almost-sure, and quantitative satisfaction for Player 1 are in NP N 
coNP. Moreover, a memoryless strategy suffices for Player 1, while Player 2 
requires an infinite memory strategy to play optimally. 
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Abstract. Emerson-Lei conditions have recently attracted attention due 
to both their succinctness and their favorable closure properties. In the 
current work, we show how infinite-duration games with Emerson-Lei 
objectives can be analyzed in two different ways. First, we show that the 
Zielonka tree of the Emerson-Lei condition naturally gives rise to a new 
reduction to parity games. This reduction, however, does not result in 
optimal analysis. Second, we show based on the first reduction (and the 
Zielonka tree) how to provide a direct fixpoint-based characterization of 
the winning region. The fixpoint-based characterization allows for sym- 
bolic analysis. It generalizes the solutions of games with known winning 
conditions such as Biichi, GR[1], parity, Streett, Rabin and Muller ob- 
jectives, and in the case of these conditions reproduces previously known 
symbolic algorithms and complexity results. 

We also show how the capabilities of the proposed algorithm can be 
exploited in reactive synthesis, suggesting a new expressive fragment of 
LTL that can be handled symbolically. Our fragment combines a safety 
specification and a liveness part. The safety part is unrestricted and 
the liveness part allows to define Emerson-Lei conditions on occurrences 
of letters. The symbolic treatment is enabled due to the simplicity of 
determinization in the case of safety languages and by using our new 
algorithm for game solving. This approach maximizes the number of 
steps solved symbolically in order to maximize the potential for efficient 
symbolic implementations. 


1 Introduction 


Infinite-duration two-player games are a strong tool that has been used, notably, 
for reactive synthesis from temporal specifications [88]. Many different winning 
conditions are considered in the literature. 

Emerson-Lei (EL) conditions were initially suggested in the context of au- 
tomata but are among the most general (regular) winning conditions considered 
for such games. They succinctly express general liveness properties by encod- 
ing Boolean combinations of events that should occur infinitely or finitely often. 
Automata and games in which acceptance or winning is defined by Emerson-Lei 
conditions have garnered attention in recent years [35/40]27/25], in particular 
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because of their succinctness and good compositionality properties (Emerson- 
Lei objectives are closed under conjunction, disjunction, and negation). In this 
work, we show how infinite-duration two-player games with Emerson-Lei winning 
conditions can be solved symbolically. 


It has been established that solving Emerson-Lei games is PSPACE-complete 
and that an exponential amount of memory may be required by winning strate- 
gies [25]. Zielonka trees are succinct tree-representations of Muller objectives [47]. 
They have been used to obtain tight bounds on the amount of memory needed 
for winning in Muller games [18], and can also be applied to analyze Emerson- 
Lei objectives and games. One indirect way to solve Emerson-Lei games is by 
transformation to equivalent parity games using later-appearance-records [25], 
and then solving the resulting parity games. Another, more recent, indirect ap- 
proach goes through Rabin games by first extracting history-deterministic Rabin 
automata from Zielonka trees and then solving the resulting Rabin games [12]. 
Both these indirect solution methods are enumerative by nature. Here, we give 
a direct symbolic algorithmic solution for Emerson-Lei games. We show how the 
Zielonka tree allows to directly encode the game as a parity game. Furthermore, 
building on this reduction, we show how to construct a fixpoint equation sys- 
tem that captures winning in the game. As usual, fixpoint equation systems are 
recipes for game solving algorithms that manipulate sets of states symbolically. 
To the best of our knowledge, we thereby give the first description of a fully 
symbolic algorithm for the solution of Emerson-Lei games. 


The algorithm that we obtain in this way is adaptive in the sense that the 
nesting structure of recursive calls is obtained directly from the Zielonka tree 
of the given winning objective. As the Zielonka tree is specific to the objective, 
this means that the algorithm performs just the fixpoint computations that are 
required for that specific objective. In particular, our algorithm instantiates to 
previously known fixpoint iteration algorithms in the case that the objective is 
a (generalized) Biichi, GR[1], parity, Streett, Rabin or Muller condition, repro- 
ducing previously known algorithms and complexity results. As we use fixpoint 
iteration, the instantiation of our algorithm to parity game solving is not di- 
rectly a quasipolynomial algorithm. In the general setting, the algorithm solves 
unrestricted Emerson-Lei games with k colors, m edges and n nodes in time 
O(k!-m-n*) and yields winning strategies with memory O(k!). 

We apply our symbolic solution of Emerson-Lei games to the automated 
construction of safe systems. The ideas of synthesis of reactive systems from 
temporal specifications go back to the early days of computer science [14]. These 
concepts were modernized and connected to linear temporal logic (LTL) and 
finite-state automata by Pnueli and Rosner [88]. In recent years, practical ap- 
plications in robotics are using this form of synthesis as part of a framework 


producing correct-by-design controllers |28]6| : 


A prominent way to extend the capacity of reasoning about state spaces is by 
reasoning symbolically about sets of states/paths. In order to apply this approach 
to reactive synthesis, different fragments of LTL that allow symbolic game anal- 
ysis have been considered. Notably, the GR[1] fragment has been widely used for 
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the applications in robotics mentioned above [877]. But also larger fragments 
are being considered and experimented with [20/9/41]. Recently, De Giacomo 
and Vardi suggested that similar advantages can be had by changing the usual 
semantics of LTL from considering infinite models to finite models (LTL +) [22]. 
The complexity of the problem remains doubly-exponential, however, symbolic 
techniques can be applied. As models are finite, it is possible to use the classical 
subset construction (in contrast to Biichi determinization), which can be rea- 
soned about symbolically. Furthermore, the resulting games have simple reach- 
ability objectives. This approach with finite models is used for applications in 
planning and robotics [6]. 

Here, we harness our symbolic solution to Emerson-Lei games to suggest a 
large fragment of LTL that can be reasoned about symbolically. We introduce the 
Safety and Emerson-Lei fragment whose formulas are conjunctions Ysafety ^ PEL 
between an (unrestricted) safety condition and an (unrestricted) Emerson-Lei 
condition defined in terms of game states. This fragment generalizes GR[1] and 
the previously mentioned works in [20/19]41]. We approach safety and Emerson- 
Lei LTL synthesis in two steps: first, consider only the safety part and convert it 
to a symbolic safety automaton; second, reason symbolically on this automaton 
by solving Emerson-Lei games using our novel symbolic algorithm. 


Psafety \ PEL ————> Do. .cory ——> Synthesis game Gy. cory AP EL 


— ae 
PEL 


(Emerson-Lei objective) 


We show that realizability of a safety and Emerson-Lei formula Ysafety ^ YEL Can 
be checked in time 20 ™%lo8 m2") where n = |Ysafety| and m = |yRy|. The overall 
procedure therefore is doubly-exponential in the size of the safety part but only 
single-exponential in the size of the liveness part; notably, both the automaton 
determinization and game solving parts can be implemented symbolically. 

We begin by recalling Emerson-Lei games and Zielonka trees in Section 
and also prove an upper bound on the size of Zielonka trees. Next we show how 
to solve Emerson-Lei games by fixpoint computation in Section [B] In Section [4] 
we formally introduce the safety and Emerson-Lei fragment of LTL and show 
how to construct symbolic games with Emerson-Lei objectives that characterize 
realizability and that can be solved using the algorithm proposed in Section 
Omitted proofs and further details can be found in the full version of this pa- 


per [23]. 


2 Emerson-Lei Games and Zielonka Trees 


We recall the basics of Emerson-Lei games [25] and Zielonka trees [47], and also 
show an apparently novel bound on the size of Zielonka trees; previously, the 
main interest was on the size of winning strategies induced by Zielonka trees, 
which is smaller [I8]. 
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Emerson-Lei games. We consider two-player games played between the exis- 
tential player 4 and its opponent, the universal player V. A game arena A = 
(V, Va, VW, E) consists of a set V = V3 W WY of nodes, partitioned into sets of 
existential nodes V3 and universal nodes YW, and a set E C V x V of moves; 
we put E(v) = {v € V | (v,v’) € E} for v € V. A play m = vovi... then is a 
sequence of nodes such that for all i > 0, (vi, vi+1) € E; we denote the set of 
plays in A by plays(A). A game G = (A, a) consists of a game arena A together 
with an objective œ C plays(A). 

A strategy for the existential player is a function o : V* - V3 — V such 
that for all r € V* and v € V3 we have (v,o(mv)) € E. A play vov: ... is said 
to be compliant with strategy f if for all i > 0 such that v; € V3 we have 
Viti = o(vo... vi). Strategy o is winning for the existential player from node 
v € V if all plays starting in v that are compliant with o are contained in q; 
then we say that the existential player wins v. We denote by W3 the winning 
region for the existential player (that is, the set of nodes that the existential 
player wins). 

In Emerson-Lei games, each node is colored by a set of colors, and the ob- 
jective a is induced by a formula that specifies combinations of colors that have 
to be visited infinitely often, or are allowed to be visited only finitely often. 
Formally, we fix a set C of colors and use Emerson-Lei formulas, that is, finite 
positive Boolean formulas y € B4 ({Inf c, Finc}-ec) over atoms of the shape Inf c 
or Finc, to define sets of plays. The satisfaction relation = for a set D C C of 
colors and an Emerson-Lei formula y (written D | y) is defined in the usual 
inductive way; D will represent the set of colors that are visited infinitely often 
by plays. E.g. the clauses for atoms Inf c and Fin c are 


DEInfesceD D HFince cé D 


Consider a game arena A = (V, V3, W, E). An Emerson-Lei condition is given 
by an Emerson-Lei formula ọ together with a coloring function y : V > 2° that 
assigns a (possibly empty) set y(v) of colors to each node v € V. The formula y 
and the coloring function y together specify the objective 


yp = {vovi ... € plays(A)|{c € C | Vi. Ij > i. c € y(v;)} E o} 


Thus a play 7 = vov: . . . is winning for the existential player (formally: 7 € a,,,) 
if and only if the set of colors that are visited infinitely often by m satisfies y. 
Below, we will also make use of parity games, denoted by (V, V3, W, E, 2) where 
N: V — {1,...,2k} (for k € N) is a priority function, assigning priorities to 
game nodes. The objective of the existential player then is that the maximal 
priority that is visited infinitely often is an even number. Parity games are an 
instance of Emerson-Lei games, obtained with set C = {p,..., pox} of colors, a 
coloring function that assigns exactly one color to each node and with objective 


Parity(pi,2:.5 P2k) = Vi oven (Inf p; A Ni<j<2k Fin p;) ‘ 


Similarly, Emerson-Lei objectives directly encode (combinations of) other stan- 
dard objectives, such as Büchi, Rabin, Streett or Muller conditions: 
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— Inf f Büchi( f) 
— Vicicg (Inf e; A Fin fi) Rabin(e1, f1,-.., €k, fk) 
— Aiicn(Fin ri V Inf gi) Streett (ri, g1,- - -Tk Jk) 
— VpeulAeep Inf c^ Ageoyp Find) Muller(U C 2°) 


Zielonka Trees. We introduce a succinct encoding of the algorithmic essence of 
Emerson-Lei objectives in the form of so-called Zielonka trees [47118]. 


Definition 1. The Zielonka tree for an Emerson-Lei formula p over set C of 
colors is a tuple Z, = (T, R,1) where (T,R C T x T) is a tree and l: T > 2° is 
a labeling function that assigns sets I(t) of colors to vertices t € T. We denote 
the root of (T, R) by r. Then Z, is defined to be the unique tree (up to reordering 
of child vertices) that satisfies the following constraints. 


— The root vertex is labeled with C, that is, I(r) = C. 
— Each vertex t has exactly one child vertex tp (labeled with (tp) = D) for 
each set D of colors that is maximal in {D' C L(t) | D! Ep & L(t) KF p}. 


For s,t E€ T such that s is an ancestor oft, we write s < t. Given a vertex s € T, 
we denote its set of direct successors by R(s) = {t € T | (s,t) € R} and the set 
of leafs below it by L(s) = {t E€ T | s < t and R(t) = 0}; we write L for the set 
of all leafs. We assume some fixed total order < on T that respects <; this order 
induces a numbering of T. A vertex t in the Zielonka tree is said to be winning 
if l(t) Æ y, and losing otherwise. We let Ty and To denote the sets of winning 
and losing vertices in Zo, respectively. Finally, we assign a level lev(t) to each 
vertex t E€ T so that lev(r) = |C|, and lev(s’) = lev(s) — 1 for all (s,s’) E€ R. 


Example 2. As mentioned above, Emerson-Lei games and Zielonka trees instanti- 
ate naturally to games with, e.g., Biichi, generalized Biichi, GR[1], parity, Rabin, 
Streett and Muller objectives; for brevity, we illustrate this for selected examples 
here (more instances can be found in [23]). 


1. Generalized Biichi condition: Given k colors f1,..., fk, the winning objective 
yp = Nieic, Inf fi expresses that all colors are visited infinitely often (not 
necessarily simultaneously); the induced Zielonka tree is depicted below with 
boxes and circles representing winning and losing vertices, respectively. 


Paya ees fi © 


2. Streett condition: The vertices in the Zielonka tree for Streett condition given 
by y = Ajeje, (Fin r; V Inf g;) are identified by duplicate-free lists L of 
colors (each entry being r; or g; for some 1 < i < k) that encode the vertex 
position in the tree. Vertex L has label /(L) = C \ L and is winning if and 
only if |L] is even. Winning vertices L have one child vertex L : gj for each 
gj € C \ L resulting in |C \ L|/2 many child vertices. Losing vertices L have 
the single child vertex L : r; where the last entry last(L) in L is gj. All leafs 
are winning and are labeled with Ø. The tree has height 2k and 2(k!) vertices. 
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3. To obtain a Zielonka tree that has branching at both winning and losing 
vertices, we consider the objective yar = (Fin a V Inf b) A ((Fin a V Fin d) A 
Inf c). This property can be seen as the conjunction of a Streett pair (a,b) 
with two disjunctive Rabin pairs (c,a) and (c,d), altogether stating that c 
occurs infinitely often and a occurs finitely often or b occurs infinitely often 
and d occurs finitely often. Below we depict the induced Zielonka tree. 


Lemma 3. The height and the branching width of Z, are bounded by |C| and 
2ICl respectively; the number of vertices in Z, is bounded by e|C|! (where e is 
Euler’s number). 


3 Solving Emerson-Lei Games 


We now show how to extract from the Zielonka tree of an Emerson-Lei objective 
a fixpoint characterization of the winning regions of an Emerson-Lei game. Solv- 
ing the game then reduces to computing the fixpoint, yielding a game solving 
algorithm that works by fixpoint iteration and hence is directly open to sym- 
bolic implementation. The algorithm is adaptive in the sense that the structure 
of its recursive calls is extracted from the Zielonka tree and hence tailored to the 
objective. As a stepping stone towards obtaining our fixpoint characterization, 
we first show how Zielonka trees can be used to reduce Emerson-Lei games to 
parity games that are structured into tree-like subgames. 

Recall that G = (V,V3,W,E,a,,,) is an Emerson-Lei game and that the 
associated Zielonka tree is Z, = (T, R,1) with set L of leaves, sets To and Tq 
of winning and losing vertices, respectively, and with root r. Following [18], we 
define the anchor vertex of v € V and t E€ T by 


anchor(v, t) = max <{s E T | s < tA q(v) Cl(s)}; 
it is the lower-most ancestor of t that contains y(v) in its label. 


A novel reduction to parity games. Intuitively, our reduction annotates nodes in 
G with leaves of Z, that act as a memory, holding information about the order 
in which colors have been visited. In the reduced game, the memory value t € L 
is updated according to a move from v to w in G by playing a subgame along 
the Zielonka tree. This subgame starts at the anchor vertex of v and t and the 
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players in turn pick child vertices, with the existential player choosing the branch 
that is taken at vertices from To and the universal player choosing at vertices 
from Tt []Once this subgame reaches a leaf t’ € L, the memory value is updated 
to t’ and another step of G is played. Due to the tree structure of Z, every play 
in the reduced game (walking through the Zielonka tree in the described way, 
repeatedly jumping from a leaf to an anchor vertex and then descending to a 
leaf again) has a unique topmost vertex from T that it visits infinitely often; 
by the definition of anchor vertices, the label of this vertex corresponds to the 
set of colors that is visited infinitely often by the according play of G. A parity 
condition can be used to decide whether this vertex is winning or losing. 

Formally, we define the parity game Pg = (V', V4, W, E’, Q), played over 
V’ = V x T, as follows. Nodes (v,t) € V’ are owned by the existential player if 
either t is not a leaf, and it is not a winning vertex (t ¢ L and t € To), or if t 
is a leaf and, in G, v is owned by the existential player (t € L and v € V3); all 
other nodes are owned by the universal player. Moves and priorities are defined 
by 


/ _ J {vu} x R(t) tL _ J 2-lev(t) tET 
R oe x {anchor(v,t)} te L a : ‘lev(#)+1 tETo 


for (v,t) € V’. Thus from (v,t) such that t is a leaf (t € L), the owner of v 
picks a move (v,w) € E and the game continues with (w,anchor(v,t)). From 
(v,t) such that t is not a leaf (t ¢ L), the owner of t picks a child t € R(t) of t 
in the Zielonka tree and the game continues with (v, t’), leaving the game node 
component v unchanged. Therefore, plays in Pg correspond to plays from G 
that are annotated with memory values t € T that are updated according to the 
colors that are visited (by moving to the anchor vertex); in addition to that, the 
owners of vertices in the Zielonka Tree are allowed to decide (by selecting one of 
the child vertices) with which colors they intend to satisfy the sub-objectives that 
are encoded by vertex labels. The priority function 2 then is used to identify the 
top-most anchor vertex s that is visited infinitely often in a play of Pg, deciding 
a play to be winning if and only if s is a winning vertex (t € Ty). We note that 
|V’| = |V|- |T| < |V|-elC|! by Lemma} 


Theorem 4. For all v € V, the existential player wins v in the Emerson-Lei 
game G if and only if the existential player wins (v,r) in the parity game Pg. 


This reduction yields a novel indirect method to solve Emerson-Lei games 
with n nodes and k colors by solving parity games with n - ek! nodes and 2k 
priorities; by itself, this reduction does not improve upon using later appearance 
records [25]. However, the game Pg consists of subgames of particular tree-like 
shapes. The remainder of this section is dedicated to showing how the special 
structure of Pg allows for direct symbolic solution by solving equivalent systems 
of fixpoint equations over V (rather than over the exponential-sized set V”). 


1 Players choose from vertices where they lose, which explains the notation Th and 
To. 
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Fixpoint equation systems. Recall (from e.g. [4]) that a hierarchical system of 
fixpoint equations is given by equations 


Xi =n, fi(X1,.--,Xk) 


for 1 < i < k, where n; € {GFP,LFP} and the f; : P(V)* + P(V) are monotone 
functions, that is, f;(A1,...,An) C fi(Bi,..., Bx) whenever A; C B; for all 
1<j<k. As we aim to use fixpoint equation systems to characterize winning 
regions of games, it is convenient to define the semantics of equation systems also 
in terms of games, as proposed in [4]. For a system S' of k fixpoint equations, 
the fixpoint game Gg = (V,V3,W,£,2) is a parity game with sets of nodes 
V3 =Vx({l,...,k} and YW = P(V)*. The set of edges E and the priority function 
N: V => {0,...,2k — 1} are defined, for (v,i) € Va and A = (Aj,..., Ak) E W, 
by 


E(v,t) ={AeEW| ve fi(A)} E(A) = {(v,i) € Va | v € Aj} 

and by Q(v,i) = 2(k — i) +1; and Q(A) = 0, where 1; = 1 if n; = LFP and ų; = 0 
if ni = GFP. We say that v is contained in the solution of variable X; (denoted 
by v € [X;]) if and only if the existential player wins the node (v,i) in Gg. 
In order to show containment of a node v in the solution of X;, the existential 
player thus has to provide a solution (Aj,...,A,) E€ W for all variables such that 
v € fi(A1,..., Ax); the universal player in turn can challenge a claimed solution 
(Ai,...,Ax) by picking some 1 < i < k and v € A; and moving to (v,i). The 
game objective checks whether the dominating equation in a play (that is, the 
equation with minimal index among the equations that are evaluated infinitely 
often in the play) is a least or a greatest fixpoint equation. 

Baldan et al. have shown in [|4| that this game characterization is equivalent to 
the more traditional Knaster-Tarski-style definition of the semantics of fixpoint 
equation systems in terms of nested fixpoints of the involved functions f;. 

To give a flavor of the close connection between fixpoint equation systems 
and winning regions in games, we recall that for a given set V of nodes, the 
controllable predecessor function CPre : 2V — 2” is defined, for X C V, by 


CPre(X) = {v € Va | E(v) NX FAO} UL{VEWU| Ev) CX}. 


Example 5. Given a Biichi game (V, V3, WY, E, Inf f) with coloring function y : 
V — 21}, the winning region of the existential player is the solution of the 
equation system 


Xı =crp X2 X2 =LFP (f N CPre(X1)) U (FA CPre(X2)) 


where f = {vE V | 7(v) ={f}} and f=V\ f. 


Our upcoming fixpoint characterization of winning regions in Emerson-Lei 
games uses the following notation that relates game nodes with anchor vertices 
in the Zielonka tree. 
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Definition 6. For a set D C C of colors, and € {C, Z} we put yp = {v € 
V | yw) x D}. For s,t E€ T such that s < t (that is, s is an ancestor of t in 
Zo), we define 


-1 =i 
anc: = Ta Tae 
where s+ is the child vertex of s that leads to t; we also put ant = ei 


Note that for fixed t € T and v € V, there is a unique s € T such that s < t 
and v € anc} (possibly, s = t); this s is the anchor vertex of t at v. 

Next, we present our fixpoint characterization of winning in Emerson-Lei 
games, noting that it closely follows the definition of Pa. 


Definition 7 (Emerson-Lei equation system). We define the system S, of 
fixpoint equations for the objective p by putting 


Urer(sy Xt R(s) # 0, s € To 


Mier) Xt R(s) 40,5 ET 
ites (anc: N CPre(Xs)) R(s)=0 


X, = 


Ns 


fors €T. For everyt €T, we use X; to refer to the variable X; where i is the 
index of t according to < and similarly for m. Furthermore, ns = GFP if s € Ti 
and ns = LFP if s € To. 


Example 8. Instantiating Definition [7] to the Biichi objective y = Inf f yields 
exactly the equation system given in Example |5| Revisiting the objectives from 
Example |2| we obtain the following fixpoint characterizations (further examples 
can be found in [23]). 


1. Generalized Biichi condition: 


X50 =GFP Ni<i<k Xs; Xs; =LFP (anc$? N CPre(X;,)) U (anc$i M CPre(X.,)) 


where anc;? = ICC N YZEL} = {v€ V | fi € y(v)} and anc; = YEON) 
2. Streett condition: 


Nozgl bia; IL] even, IL] < 2k 
XL =n Xi:r; [L] odd, last(L) 95 
(ancl N CPre(Xq)) U...U (ancl A CPre(X1)) |L| = 2k 


where ņ = GFP if |L| is even and ņ, = LFP if |L| is odd. Here, ancff = 
TeO\K N YIZO for K # L and J = Kı, and anc} = yc, both for L such that 
|L| = 2k. 

3. The equation system associated to the Zielonka tree for the complex objective 


PEL from Example|2}3 is as follows, where we use a formula over the colors 
to denote the set of vertices whose label satisfies the formula. For example, 
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b A wd corresponds to vertices whose set of colors contains b but does not 
contain d. 


Xı =irp X2 U X3 X2 =cFP X4 N X5 X3 =cFP X6 Xs =irp X7 X7 =crp Xs 
Xa =ıFp (ac A ad N Cpre(X4)) U (c A ad Cpre(X2)) U (d N Cpre(X1)) 
Xe =ıfp (ma A ac NM Cpre(X6)) U (~a A cN Cpre(X3)) U (a N Cpre(X1)) 
Xs =p (ma A ab Anac A adN Cpre(Xg)) U (ma A mb Ac A ad Cpre(X7)) U 
(a nnb A Ad N Cpre(X5)) U (b A ad NM Cpre(X2)) U (d N Cpre(X1)), 


Theorem 9. Referring to the equation system from Definition [] and recalling 
that r is the root of the Zielonka tree Zo, the solution of the variable X, is the 
winning region of the existential player in the Emerson-Lei game G. 


By Theorem|4| it suffices to mutually transform winning strategies in Pg and 
the fixpoint game G's, for the equation system Sy from Definition [7] 

Given the fixpoint characterization of winning regions in Emerson-Lei games 
with objective y in Definition [7] we obtain a fixpoint iteration algorithm that 
computes the solution of Emerson-Lei games. The algorithm is by nature open 
to symbolic implementation. The main function is recursive, taking as input one 
vertex s € T of the Zielonka tree Z, and a list | of subsets of the set V of nodes, 
and returns a subset of V as result. For calls SOLVE(s,/s), we require that the 
argument list ls contains exactly one subset Xy of V for each ancestor s’ of s 
in the Zielonka tree (with s’ < s). 


Algorithm 1 SOLVE(s,/s) 


if s € To then X, + 0 else X, V > Initialize variable X, for lfp/gfp 
W+V\Xs 
while X, 4 W do > Compute fixpoint 
W+Xs 
if R(s) # Ø then > Case: s is not a leaf in Z% 
for t € R(s) do 
U + SoLtvE(t, ls : W) > Recursively solve for t 


if s € To then Xs + X, UU 
else Xs + Xs QAU 
end for 
else > Case: s is a leaf in Z, 
Yeo 
for t < s do 
U < anc, CPre((ls : W)(t)) > Compute one-step attraction w.r.t. s 
YeYUU 
end for 
Xs Y 
end if 
end while 
return X, >œ Return stabilized set X, as result 
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Lemma 10. For allv E€ V, we have v € [X,] if and only if v € SOLVE(r, []). 


Proof (Sketch). The algorithm computes the solution of the equation system by 
standard Kleene-approximation for nested least and greatest fixpoints. 


Lemma 11. Given an Emerson-Lei game (V,V3,W, E, &y, p) with set of colors 
C and induced Zielonka tree Z,, the solution [X,] of the equation system Sọ 
from Definition [7] can be computed in time O(|Z,|-|E|-|V|*), where k < |C] 
denotes the height of Zo. 


Combining Theorem p] with Lemmas and |11|we obtain 


Corollary 12. Solving Emerson-Lei games with n nodes, m edges and k colors 
can be implemented symbolically to run in time O(k!-m-n*); the resulting 
strategies require memory at most e- k!. 


Remark 18. Strategy extraction works as follows. The algorithm computes a set 
[X:+] for each Zielonka tree vertex t € Z,. Furthermore it yields, for each non- 
leaf vertex s € To and each v € [X,], a single child vertex choice(v, s) € R(s) 
of s such that v € [Xchoice(v,s)]. The algorithm also yields, for each leaf vertex t 
and each v € VaN[X;], a single game move move(v, t). All these choices together 
constitute a winning strategy for existential player in the parity game Pg. We 
define a strategy for the Emerson-Lei game that uses leaves of the Zielonka 
tree as memory values, following the ideas used in the construction of Pg; the 
strategy moves, from a node v € V3 and having memory content m, to the 
node move(v,m). As initial memory value we pick some leaf of Z, that choice 
associates with the initial node in G. To update memory value m according to 
visiting game node v, we first take the anchor vertex s of m and v. Then we pick 
the next memory value m to be some leaf below s that can be reached by talking 
the choices choice(v, s’) for every vertex s € To passed along the way from s 
to the leaf; if s € Tp, then we additionally require the following: let q = |R(s)|, 
let o be the number such that m is a leaf below the o-th child of s, and put 
j =o0+1 mod q; then we require that m’ is a leaf below the j-th child of s. By 
the correctness of the algorithm, the constructed strategy is a winning strategy. 

Dziembowski et al. have shown that winning strategies can be extracted by 
using a walk through the Zielonka tree that requires memory only for the branch- 
ing at winning vertices [18]. This yields, for instance, memoryless strategies for 
games with Rabin objectives, for which branching in the associated Zielonka 
trees takes place at losing vertices. Adapting the strategy extraction in our set- 
ting to this more economic method is straight-forward but notation-heavy, so we 
omit a more precise analysis of strategy size here. 


Our algorithm hence can be implemented to run in time 20's”) for games 


with n nodes and k < n colors, improving upon the bound 200°) stated in [25], 
where the authors only consider the case that every game node has a distinct 
color, implying n = k. We note that the later appearance record construction 
used in is known to be hard to represent symbolically. Our fixpoint charac- 
terization generalizes previously known algorithms for e.g. parity games [8], and 
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Streett and Rabin games [36], recovering previously known bounds on worst-case 
running time of fixpoint iteration algorithms for these types of games. 

While it has recently been shown that parity games can be solved in quasipoly- 
nomial time [9], we note that in the case of parity objectives, our algorithm is not 
immediately quasipolynomial. However, there are quasipolynomial methods for 
solving nested fixpoints (with the latter being open to symbolic implemen- 
tation); in the case of parity objectives, these more involved algorithms can be 
used in place of fixpoint iteration to solve our equation system and recover the 
quasipolynomial bound. The precise complexity of using quasipolynomial meth- 
ods for solving fixpoint equation systems beyond parity conditions is subject to 
ongoing research. 


4 Synthesis for Safety and Emerson-Lei LTL 


In this section we present an application of the results from Section |3| We in- 
troduce the safety and Emerson-Lei fragment of LTL and show that synthesis 
for this fragment can be reasoned about symbolically. The idea for safety and 
Emerson-Lei LTL synthesis is twofold: first, consider only the safety part and 
create a symbolic arena capturing its satisfaction. Second, play a game on this 
arena by adding the Emerson-Lei part as a winning condition. Finally we use 
the results from the previous sections to solve the game symbolically. 


4.1 Safety LTL and Symbolic Safety Automata 


We start by defining safety LTL, symbolic safety automata, and recalling known 
results about those. 


Definition 14 (LTL and Safety LTL [45]). Given a non-empty set AP of 
atomic propositions, the general syntax for LTL formulas is as follows: 


g:=Tl|Llpl-el gi Aga| eV p2 | XY | p1U p2 p€ AP. 


Standard abbreviations are defined as follows: pı Rp2 := 7(>y1U7¢2), Fy := 
TU, and Gy := =F 7. We define the satisfaction relation = for a formula p 
and its language L(y) as usual. 

An LTL formula is said to be a safety formula if it is in negative normal form 
(i.e. all negations are pushed to atomic propositions) and only uses X,R,G as 
temporal operators (i.e. no U or F are allowed). 


It is a safety formula in the sense that every word that does not satisfy the 
formula has a finite prefix that already falsifies the formula. In other words, such 
a formula is satisfied as long as “bad states” are avoided forever. 


Definition 15 (Symbolic Safety Automata). A symbolic safety automaton 
is a tuple A = (24°, V, T, 0o) where V is a set of variables, T(V,V'’,AP) is the 


transition assertion, and 09(V) is the initialization assertion. A run of A on 
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the word w € (24°) is a sequence p = 8981... where the s; € 2V are variable 
assignments such that 1. so = 0o, and 2. for alli > 0, (si, Si41, w(t)) = T. 
A word w is in L(A) if and only if there is an infinite run of A on w. A is 
deterministic if for all words w € (24°) there is at most one run of A on w. 


Kupferman and Vardi show how to convert a safety LTL formula into an 
equivalent deterministic symbolic safety automaton [30]. 


Lemma 16. A safety LTL formula p can be translated to a deterministic sym- 
bolic safety automaton Dsymp accepting the same language, with |Dsymb| = all, 


The idea is to first convert y to a (non-symbolic) non-deterministic safety 
automaton Mg, which is of size exponential of the size of the formula, and then 
symbolically determinize Mọ by a standard subset construction to obtain Dsymb- 
Note that while the size of Dsymb is only exponential in the size of the formula, 
its state space would be double exponential when fully expanded. 


Example 17. Let y = G(b V c) \G(a > bV X Xb) be a safety LTL formula over 
AP = {a,b,c}. An execution satisfying p must have at least one of b or c at every 
step, moreover every a sees a b present at the same step or two steps afterwards. 

As an intermediate step towards building the equivalent Dsymb, we first 
present below a corresponding non-deterministic safety automaton Mọ. 


~a Vb b 


start 


For the sake of presentation, we use Boolean combinations of AP in transitions 
instead of labeling them with elements of 2^P, with the intended meaning that 


s $ s = {s s| C e2, C H y}. We also omit the G(b Vc) part of 
the formula in the construction. One can simply append --- A (b V c) to every 
transition of Mọ to get back the original formula. Intuitively state 1 correspond 
to not seeing an a, state 2 means that a b must be seen at the next step, state 
3 means that there must be a b now, and state 4 that b is needed now and next 
as well. 

Then the symbolic safety automaton is Dsymb = (24°, V, T, 0o) with: 


— V = {v1, v2, v3, v4} are the variables corresponding to the four states of Mọ, 
Oo = vy A 72 A 73 A ~w asserts that only the state v1 is initial, 

— The transition assertion is T = (vi © (vı A (“a V b)) V (v3 A b)) A 
(us © (vi Aa) V (v3 A (aA b))) A (05 © (v2 A (~a V b)) V (v4 A b)) A 
(v4 e (v2 Aa) V (v4 A (a A b))) A (v1 V v2 V v3 V v4). 


Determinizing M, enumeratively would give an automaton with 9 states (see 
Example |23). 
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Remark 18. Restricting attention to safety LTL enables the two advantages men- 
tioned above with respect to determinization. First, subset construction suffices 
(as observed also in [46]), avoiding the more complex Biichi determinization. 
Second, this construction, due to its simplicity, can be implemented symboli- 
cally. Interestingly, recent implementations of the synthesis from LTL ș or 
from safety LTL have used indirect approaches for obtaining deterministic 
automata. For example, by translating LTL to first order logic and applying the 
tool MONA to the results [45/46], or by concentrating on minimization of de- 
terministic automata [42]. The direct construction is similar to approaches used 
for checking universality of nondeterministic finite automata or SAT-based 
bounded model checking [I]. We are not aware of uses of this direct implementa- 
tion of the subset construction in reactive synthesis. The worst case complexity 
of this part is doubly-exponential, which, just like for LTL and LTLy, cannot be 


avoided [43]3}. 


4.2 Symbolic Games 


We use symbolic game structures to represent a certain class of games. Formally, 
a symbolic game structure G = (V, X,Y, 03, p3, p) consists of: 


e V = {v1,...,Un}:A finite set of typed variables over finite domains. Without 

loss of generality, we assume they are all Boolean. A node s is an valuation 
of V, assigning to each variable v; € V a value s[v;] € {0,1}. Let X be the 
set of nodes. 
We extend the evaluation function s[-] to Boolean expressions over V in the 
usual way. An assertion is a Boolean formula over V. A node s satisfies an 
assertion y denoted s = y, if s[y] = true. We say that s is a y-node if 
sEy. 

e X C Vis a set of input variables. These are variables controlled by the 
universal player. Let Xæ denote the possible valuations to variables in 7. 

e y = V \ X isa set of output variables. These are variables controlled by the 
existential player. Let Xy denote the possible valuations to variables in ). 

e 03(%,Y) is an assertion characterizing the initial condition. 

e pa(V, X’, Y’) is the transition relation. This is an assertion relating a node 
s € X and an input value sy E€ Xx to an output value sy € Xy by referring 
to primed and unprimed copies of V. The transition relation p3 identifies 
a valuation sy E€ Xy as a possible output in node s reading input sæ if 
(s, (sx,sy)) H pa, where s is the assignment to variables in V and sy and 
sy are the assignment to variables in V’ induced by (sy, sy) € X. 

e vy is the winning condition, given by an LTL formula. 


For two nodes s and s’ of G, s’ is a successor of s if (s, s') = pa. 
A symbolic game structure G defines an arena Ag, where WỌ = YX, Va = 
X x Xx, and E is defined as follows: 


E = {(s, (s,sx)) | s E€ X and sx € Xx fU{((s, sx), (8x, sy)) | (s, (8x, 8y)) E pat. 
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When reasoning about symbolic game structures we ignore the intermediate 
visits to V3. Indeed, they add no information as they can be deduced from the 
nodes in Vy preceding and following them. Thus, a play m = s951... is winning 
for the existential player if o is infinite and satisfies y. Otherwise, o is winning 
for the universal player. 

The notion of strategy and winning region is trivially generalized from games 
to symbolic game structures. When needed, we treat Wa (the set of nodes win- 
ning for the existential player) as an assertion. We define winning in the entire 
game structure by incorporating the initial assertion: a game structure G is said 
to be won by the existential player, if for all sy E€ Xx there exists sy E€ Xy such 
that (sx, sy) E 65 A W3. 


4.3 Realizability and Synthesis 


Let y be an LTL formula over input and output variables J and O, controlled by 
the environment and the system, respectively (the universal and the existential 
player, respectively). 

The reactive synthesis problem asks whether there is a strategy for the system 
of the form ø : (24)+ — 2° such that for all sequences ax +- € (21)” we have: 


(ap U o(x0))(a1 U o(xoz1))... Ep 


If there is such a strategy we say that ọ is realizable [83]. 
Equivalently, y is realizable if the system is winning in the symbolic game 
G, = (IU O,1,O,T,T,p) with I for input variables ¥ and O for output YV. 


Theorem 19. [99] Given an LTL formula ¢, the realizability of p can be de- 
termined in doubly exponential time. The problem is 2EXPTIME-complete. 


The game G, above uses neither the initial condition nor the system transi- 
tion. Conversely, consider a symbolic game G = (V, ¥, V, 03, pa, P): 


Theorem 20. [7 The system wins in G iff pg = 03 \Gpa\¢ is realizable 


4.4 Safety and Emerson-Lei Synthesis 


We now define the class of LTL formulas that are supported by our technique and 
show how to construct appropriate games capturing their realizability problem. 

For y € B(AP), let Ify := GFw and Finy := FGry = alnfy. The 
Emerson-Lei fragment of LTL consists of all formulas that are positive Boolean 
combinations of Inf Yy and Finw for all Boolean formulas p over atomic proposi- 
tions. The satisfaction of such formulas depends only on the set of letters (truth 
assignments to propositions) appearing infinitely often in a word. 


2 Technically, p3 contains primed variables and is not an LTL formula. This can be 
easily handled by using the next operator X. We thus ignore this issue. 

3 We note that Bloem et al. consider more general games, where the environment also 
has an initial assertion and a transition relation. Our games are obtained from theirs 
by setting the initial assertion and the transition relation of the environment to true. 


70 D. Hausmann et al. 


Remark 21. The Emerson-Lei fragment easily accommodates various liveness 
properties that cannot be encoded in smaller fragments such as GR[1]. One 
prominent example for this is the property of stability (as encoded by LTL 
formulas of the shape FG p), which appears frequently as a guarantee in us- 
age of synthesis for robotics and control (see, e.g., the work of Ehlers and 
Ozay [32]), and commonly is approximated in GR[1] but, as a guarantee or as 
part of a specification, cannot be captured exactly in the game context. Another 
important example is strong fairness (as encoded by LTL formulas of the shape 
A,(GF ri + GF g;)) which allows to capture the exact relation between cause 
and effect. Particularly, in GR[1] only if all “resources” are available infinitely 
often there is an obligation on the system to supply all its “guarantees”. In 
contrast, strong fairness allows to connect particular resources to particular sup- 
plied guarantees. Ongoing studies on fairness assumptions that arise from the 
abstraction of continuous state spaces to discrete state spaces [32/33] provide 
further examples of fairness assumptions that can be expressed in EL but not in 
GR[1]. Emerson-Lei liveness allows free combination of all properties mentioned 
above and more. 


Definition 22. The Safety and Emerson-Lei fragment is the set of formulas of 
the form p = Ysafety \ PEL, Where Psafety is a safety formula and pp, is in the 
Emerson-Lei fragment. 


We assume a partition AP = I W O where I is a set of input propositions 
and O a set of output propositions, both non-empty. Let Y = YPsafety \ PEL be 
a safety and Emerson-Lei formula over AP, and let Dymo = (24°,V,T, 0) be 
the symbolic deterministic safety automaton associated to Ysafety. We construct 
Gy = (V W AP, I, O W V, 0o, T, peL), thus ¥ =I and Y = O WV. 


Example 23. Let Ysatety = G(b V c) A G(a > b V XXb), our running safety 
example from Example[17| with its associated symbolic deterministic automaton. 
Partition AP into J = {a} and O = {b,c}. We depict the arena of the game G, 
(independent of the formula ppr that is yet to be defined) in Figure [i] 

To keep the illustration readable and keep it from getting too large, a few 
modifications to the formal arena definition have been made. First, c labels on 
edges have been omitted: every transition labeled with b represent two transitions 
with sets {b} and {b,c}, while transitions labeled with ~b stand for a single 
transition with set {c} (due to the G(bV c) requirement forbidding Ø). Similarly, 
existential nodes have been omitted when all choices for the existential player 
lead to the same destination. Instead, the universal and existential moves have 
been combined in one transition: a; * for an a followed by some existential move, 
and a;b for when an a requires the existential player to play b (with or without 
c, as above). Finally, states are only labeled with variables from V and not AP, 
the latter is used to label edges instead. For a fully state-based labeling arena, 
states would have to store the last move, leading to various duplicate states. 

Note that this game arena is given only for illustration purposes, as we want 
to solve the symbolic game without explicitly enumerating all its states and 
transitions like here. 
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Fig. 1. Game arena for Go 


Lemma 24. The system wins Gy if and only if p is realizable. 


Next we detail how to solve the symbolic game Go by using the results from 
Section B] 


Lemma 25. Given a symbolic game G = (V, X,Y, 63, pa, p} such that p is an 
Emerson-Lei formula with set of colors 


C = {w € B(AP) | y is a subformula of p}, 


the winning region Wa of G is characterized by the equation system from Defi- 
nition [) using the assertion 


CPre(S) = Ysy € Xx. Isy € Xy. S'A (v, sx, sy) E pa. 


The proof of this lemma is by straightforward adaptation of the proof of 
Theorem [9]to the symbolic setting, following the relation between symbolic game 
structures and game arenas described above. 

Finally, this gives us a procedure to solve the synthesis problem for safety 
and Emerson-Lei LTL. 


Theorem 26. The realizability of a formula ~ = Qsatety \ PEL of the Safety 
and Emerson-Lei fragment of LTL can be checked in time 200 18™2") | where 
n = |Psafety| and m = |yrz|. Realizable formulas can be realized by systems of 
size at most 2?” -e-ml. 


Proof. Using the construction described in this section,we obtain the symbolic 
game Go of size q = 2?" with winning condition ypz, using at most m colors; by 
Theorem[24] this game characterizes realizibility of the formula. Using the results 
from the previous section, G can be solved in time O(m! -q?-q™) € O(2m 98m.. 
gim+2)2") © gO(mlogm2") resulting in winning strategies with memory at most 
e-ml. 
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Both the automata determinization and the game solving can be implemented 
symbolically. 


Example 27. To illustrate the overall synthesis method, we consider the game 
that is obtained by combining the game arena Gogy from pa with the 
winning objective yer = (Fin aV Inf b) A (Fin a V Find) A Inf c from xample[2]3, 
where we instantiate the label d to nodes satisfying b A c thus creating a game- 
specific dependency between the colors. Solving this game amounts to solving 
the equation system shown in Example 33. However, with the interpretation 
of d = b A c, some of the conditions become simpler. For example, ~a ^A =b A 
ac A 7d becomes ~na ^A ab A ac and b A ad becomes b A ~c. It turns out that the 
system player wins the node vı. Intuitively, the system can play {c} whenever 
possible and thereby guarantee satisfaction of yer. We extract this strategy 
from the computed solution of the equation system in Example p]3 as described 
in Remark E.g. for partial runs m that end in vı and for which the last 
leaf vertex in the induced walk p, through Z, is the vertex 8, the system can 
react by playing {b}, {c}, or even {b,c} whenever the environment plays Ø. The 
move {b} continues the induced walk py through vertex 2 to the leaf vertex 5; 
similarly, the move {b,c} continues p, through the vertex 1 to the leaf vertex 6. 
The strategy construction gives precedence to the choice that leads through the 
lowest vertex in the Zielonka tree, which in this case means picking the move 
{c} that continues p, through the vertex 7 to the leaf 8. Proceeding similarly 
for all other combinations of game nodes and vertices in the Zielonka tree, one 
obtains a strategy o for the system that always outputs singleton letters, giving 
precedence to {c} whenever possible. To see that ø is a winning strategy, let m 
be a play that is compatible with øo. If 7 eventually loops at vı forever, then s+ is 
the existential vertex 7 and the existential player wins the play since it satisfies 
both Fin a and Inf c. Any other play m satisfies Inf a, Inf b and Inf c since all 
cycles that are compatible with ø (excluding the loop at v1) contain at least one 
a-edge, at least one b-edge and also at least one c-edge that is prescribed by the 
strategy o. For these plays, pr eventually reaches the vertex 2. Since the system 
always plays singleton letters (so that m in particular satisfies Fin(b A c)), the 
vertex 1 is not visited again by px, once vertex 2 has been reached. Hence the 
dominating vertex for such plays is s; = 2, an existential vertex. 


4.5 Synthesis Extensions and Optimizations 


We have chosen to use safety-LTL as the safety part of the Safety-EL fragment 
to showcase the options opened by having symbolic algorithms for the analysis 
of very expressive liveness conditions. The crucial feature of the safety fragment 
is the ability to convert that part of the specification to a symbolic deterministic 
automaton. It is important to note that every fragment of LTL (or w-regular 
in general) that can be easily converted to a symbolic deterministic automaton 
can be incorporated and handled with the same machinery. For example, it was 
suggested to extend the expressiveness of GR{1] by including deterministic au- 
tomata in the safety part of the game and referring to their states in the liveness 
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part [7]. Past LTL [31] can be handled in the same way in that it is incorporated 
for GR[1] [7]. An extreme example is GR-EBR, where safety parts are allowed to 
use bounded future and pure past, which still allows the symbolic treatment [15]. 
All of these alternatives can be incorporated in the safety part with no changes 
to our overall methodology. Unlike previous cases, if there is an easy translation 
to deterministic symbolic automata with a non-trivial winning condition, these 
can be incorporated as well with the EL part extended to handle their winning 
condition as well. We could consider also extensions to the liveness parts. For 
example, by using past LTL or reference to states of additional symbolic de- 
terministic automata. The Boolean state formulas appearing as part of the EL 
condition can be replaced by formulas allowing one usage of the next operator, as 
in [39]19]. The generalization to handle transition-based EL games, which would 
be required in that case, rather than state-based EL games is straight-forward. 

As the formulas we consider are conjunctions, optimizations can be applied 
to both conjuncts independently. This subsumes, for example, analyzing the win- 
ning region in a safety game prior to the full analysis [29[7J5], reductions in the 
size of nondeterministic automata [I7], or symbolic minimization of deterministic 


automata [16] j] 


5 Conclusions and Future Work 


We provide a symbolic algorithm to solve games with Emerson-Lei winning con- 
ditions. Our solution is based on an encoding of the Zielonka tree of the winning 
condition in a system of fixpoint equations. In case of known winning conditions, 
our algorithm recovers known algorithms and complexity results. As an appli- 
cation of this algorithm, we suggest an expressive fragment of LTL for which 
realizability can be reasoned about symbolically. Formulas in our fragment are 
conjunctions between an LTL safety formula and an Emerson-Lei liveness con- 
dition. This fragment is more general than, e.g., GR[1]. 

In the future, we believe that analysis of the Emerson-Lei part can reduce the 
size of Zielonka trees (and thus the symbolic algorithm). This can be done either 
through analysis and simplification of the LTL formula, e.g., [26], by means of 
alternating-cycle decomposition , or by analyzing the semantic meaning of 
colors. We would also like to implement the proposed overall synthesis method. 
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Abstract. Temporal graphs are a popular modelling mechanism for dy- 
namic complex systems that extend ordinary graphs with discrete time. 
Simply put, time progresses one unit per step and the availability of 
edges can change with time. 

We consider the complexity of solving w-regular games played on tem- 
poral graphs where the edge availability is ultimately periodic and fixed 
a priori. 

We show that solving parity games on temporal graphs is decidable in 
PSPACE, only assuming the edge predicate itself is in PSPACE. A match- 
ing lower bound already holds for what we call punctual reachability 
games on static graphs, where one player wants to reach the target at a 
given, binary encoded, point in time. We further study syntactic restric- 
tions that imply more efficient procedures. In particular, if the edge pred- 
icate is in P and is monotonically increasing for one player and decreasing 
for the other, then the complexity of solving games is only polynomially 
increased compared to static graphs. 


Keywords: Temporal graphs - Reachability Games - Complexity - Timed 
automata 


1 Introduction 


Temporal graphs are graphs where the edge relation changes over time. They 
are often presented as a sequence Go, G1,... of graphs over the same set of 
vertices. We find it convenient to define them as pairs G = (V, E) consisting 
of a set V of vertices and associated edge availability predicate E : V? > 2N 
that determines at which integral times a directed edge can be traversed. This 
model has been used to analyse dynamic networks and distributed systems in dy- 
namic topologies, such as gossiping and information dissemination B624]. There 
is also a large body of work that considers temporal generalisations of various 
graph-theoretic notions and properties [32] [4][0]} Related algorithmic questions 
include graph colouring [80], exploration [[2], travelling salesman [83], maxi- 
mum matching [29], and vertex-cover [2]. The edge relation is often deliberately 
left unspecified and sometimes only assumed to satisfy some weak assumptions 
about connectedness, frequency, or fairness to study the worst or average cases 
in uncontrollable environments. Depending on the application, one distinguishes 
between “online” questions, where the edge availability is revealed stepwise, as 
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opposed to the “offline” variant where all is given in advance. We refer to |L7.BI]] 
for overviews of temporal graph theory and its applications. 

Two player zero-sum verification games on directed graphs play a central role 
in formal verification, specifically the reactive synthesis approach [84] Here, a 
controllable system and an antagonistic environment are modeled as a game in 
which two opposing players jointly move a token through a graph. States are 
either owned by Player 1 (the system) or Player 2 (the environment), and the 
owner of the current state picks a valid successor. Such a play is won by Player 1 
if, and only if, the constructed path satisfies a predetermined winning condition 
that models the desired correctness specification. The winning condition is of- 
ten given either in a temporal logic such as Linear Temporal Logic (LTL) [B5] 
or directly as w-automaton whose language is the set of infinite paths consid- 
ered winning for Player 1. The core algorithmic problem is solving games: to 
determine which player has a strategy to force a win, and if so, how. 

Determining the complexity of solving games on static graphs has a long 
history and continues to be an active area of research. We refer to [LIB] for 
introductions on the topic and recall here only that solving reachability games, 
where Player 1 aims to eventually reach a designated target state, is complete 
for polynomial time. The precise complexity of solving parity games is a long- 
standing open question. It is known to be in UPNcoUP [R2], and so in particular in 
NP and coNP, and recent advances have led to quasi-polynomial time algorithms 


[623.D6.p.B5]) 


Related Work. Periodic temporal graphs were first studied by Floccchini, Mans, 
and Santoro in [{[4],| where they show polynomial bounds on the length of ex- 
plorations (paths covering all vertices). Recently, De Carufel, Flocchini, Santoro, 
and Simard [[[0]]|study Cops & Robber games on periodic temporal graphs. They 
provide an algorithm for solving one-cop games that is only quadratic in the 
number of vertices and linear in the period. 

Games on temporal graphs with maximal age, or period of some absolute 
value K given in binary are games on exponentially succinctly presented arenas. 
Unfolding them up to time K yields an ordinary game on the exponential sized 
graph which allows to transfer upper bounds, that are not necessarily optimal. 
In a similar vein, Avni, Ghorpade, and Guha [f]] have recently introduced types 
of games on exponentially succinct arenas called pawn games. Similar to our 
results, their findings provide improved PSPACE upper bounds for reachability 
games. 

Parity games on temporal graphs are closely related to timed-parity games, 
which are played on the configuration graphs of timed automata [8]| However, the 
time in temporal graphs is discrete as opposed to the continuous time semantics 
in timed automata. Solving timed parity games is complete for EXP[28.B]] and 
the lower bound already holds for reachability games on timed automata with 
only two clocks [21], Unfortunately, a direct translation of (games on) temporal 
graphs to equivalent timed automata games requires at least two clocks: one to 
hold the global time used to check the edge predicate and one to ensure that 
time progresses one unit per step. 
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Contributions. We study the complexity of solving parity games on temporal 
graphs. As a central variant of independent interest are what we call punctual 
reachability games, that are played on a static graph and player wants to reach a 
target vertex at a given binary encoded time. We show that solving such games 
is already hard for PSPACE, which provides a lower bound for all temporal graph 
games we consider. 

As our second, and main result, we show how to solve parity games on (ul- 
timately) periodic temporal graphs. The difficulty to overcome here is that the 
period may be exponential in the number of vertices and thus a naively solv- 
ing the game on the unfolding only yields algorithms in exponential space. Our 
approach relies on the existence of polynomially sized summaries that can be 
verified in PSPACE using punctual reachability games. 

We then provide a sufficient syntactic restriction that avoids an increased 
complexity for game solving. In particular, if the edge predicate is in polynomial 
time and is monotonically increasing for one player and decreasing for the other, 
then the cost of solving reachability or parity games on temporal graphs increases 
only polynomially in the number of vertices compared to the cost of solving these 
games on static graphs. 


None of our upper bounds rely on any particular representation of the edge 
predicate. Instead, we only require that the representation ensures that checking 
membership (if an edge is traversable at a given time) has suitably low com- 
plexity. That is, our approach to solve parity games only requires that the edge 
predicate is in PSPACE, and polynomial-time verifiable edge predicates suffice 
to derive P-time upper bounds for monotone reachability games. These condi- 
tions are met for example if the edge predicate is defined as semilinear set given 
as an explicit union of linear sets (NP in general and in P for singleton sets of 
periods), or by restricted Presburger formulae: the quantifier-free fragment is in 
P, the existential fragment is in NP but remains in P if the number of variables 
is bounded [37]. See for instance [15] and contained references. 


The rest of the paper is structured as follows. We recall the necessary no- 
tations in Section Bland then discuss reachability games in Section B] Section f] 
presents the main construction for solving parity games and finally, in Section [5] 
we discuss improved upper bounds for monotone temporal graphs. 


2 Preliminaries 


Definition 1 (Temporal Graphs). A temporal graph G = (V, E) is a directed 
graph where V are vertices and E : V? — 2N is the edge availability relation that 
maps each pair of vertices to the set of times at which the respective directed edge 


can be traversed. If i € E(s,t) we call t an i-successor of s and write s 5t. 

The horizon of a temporal graph is h(G) = sup, tey (E(s,t)), the largest 
finite time at which any edge is available, or oo if no such finite time exists. 
A temporal graph is finite if h(G) € N i.e., every edge eventually disappears 
forever. A temporal graph is periodic with period K € N if for all nodes s,t € V 
it holds that E(s,t) = E(s,t) + K-N. We call G static if it has period 1. 
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Naturally, one can unfold a temporal graph into its expansion up to some 
time T € NU {oo}, which is the graph with nodes V x {0,1,..., 7} and directed 
edges (s,i) > (t,i + 1) iff i € E(s,t). 

In order for algorithmic questions to be interesting, we assume that temporal 
graphs are given in a format that is more succinct than the expansion up to their 
horizon or period. We only require that the representation ensures that checking 
if an edge is traversable at a given time can be done reasonably efficiently. 

We will henceforth use formulae in the existential fragment of Presburger 
arithmetic, the first-order theory over natural numbers with equality and addi- 
tion. That is, the JPA formula s +(x) with one free variable x represents the set 
of times at which an edge from s to tis available as E(s,t) = {n | Bs (n) = true}. 
We use common syntactic sugar including inequality and multiplication with (bi- 


nary encoded) constants. For instance, Ps +(x) “5 < rAr < 10 means the edge 
def + 


is available at times {5,6,7,8,9, 10}; and Bs (£) = dy.(w = y- 7) A7(ax < 100) 
means multiples of 7 greater than 100. 


Definition 2 (Parity Games). A parity game is a zero-sum game played by 
two opposing players on a directed graph. Formally, the game is given by a game 
graph G = (V, E), a partitioning V = Vı © Vz of vertices into those owned by 
Player 1 and Player 2 respectively, and a colouring col: V —> C of vertices into 
a finite set C CN of colours. 

The game starts with a token on an initial vertex so € V and proceeds in turns 
where in round i, the owner of the vertex occupied by the token moves it to some 
successor. This way both players jointly agree on an infinite path p = 8981... 
called a play. A play is winning for Player 1 if max{c | Vidj.col(s;) = c}, the 
maximum colour seen infinitely often, is even. 


A strategy for Player i is a recipe for how to move. Formally, it is a function 
ci : V*V; > V from finite paths ending in a vertex s in V; to some successor. We 
call o positional if o(7s) = o(n's) for any two prefixes n,n’ E€ V*. A strategy 
is winning from vertex s if Player i wins every play that starts in vertex s and 
during which all decisions are made according to o. 


We call a vertex s winning for Player i if there exists a winning strategy from 
s, and call the subset of all such vertices the winning region for that player. 
Parity games enjoy the following property (See Theorem 15] for details). 


Proposition 1. Parity games are uniformly positionally determined: For every 
game (V =V1 WV, Æ, col) there is a pair 01,02 of positional strategies so that c; 
is winning for Player i from every vertex in the winning region of Player i. 


A temporal parity game is a parity game played on the infinite expansion of 
a temporal graph G = (V, E), where the ownership and colouring of vertices are 
given with respect to the underlying directed graph V = V WV and col: V > C. 
The ownership and colouring are lifted to the expansion so that vertices in V; x N 
are owned by Player i and vertex (s,n) has colour col(s). 
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Fig. 1: An example of a temporal parity game. Player 1 controls the diamond 
vertices Vj = {s,v} and Player 2 controls square vertices Vz = {r,t,u,w}. Edge 
labels are Presburger formulae constraints denoting when an edge is available; 
edges without constraints are always available. The grey label next to each node 
denotes its colour. E.g., col(s) = 1 € C = {1,2,3,4}. 


Example 1. Consider the temporal parity game shown in Fig.}1} We will draw 
Player 1 states as diamond and those controlled by Player 2 as squares and 
sometimes write modulo expressions to define the edge availability. For example, 
the constraint on the edge from u to v can be written as the JPA-formula as 
Jy.(x = 3y) V (x = 3y + 1) and so this edge is available at times 0,1,3,4,6,.... 
The temporal graph underlying this game has period 15. 

Player 1 has a winning strategy starting from (s,i) in the expansion by 
staying in state s until time 7’ > i with 7’ = 0 mod 5 and then following the 
edge to (t, i' +1). If Player 2 ever chooses to move to r, he is trapped in an even- 
coloured cycle; if he stays in t forever, then the resulting game sees only colour 
2 and is losing for him. Otherwise, if the game continues at (s, i’ + 2), Player 1 
repeats as above (and wins plays that see both states s and t. The example 
shows that Player 1 s strategies depend on the time and are not positional in 
the vertices alone, even if the winning set has period 1. Indeed, the only possible 
vertex-positional strategy (cycle in s) is losing. 


The vertices {s,t} shaded in blue represent the vertex from which Player 1 
can win starting at any time, following the strategy described above. From the 
vertices shaded in red, Player 2 can win starting at certain times. For exam- 
ple, Player 2 has a winning strategy from (u,i) if, and only if, 7 = 0 mod 3 
or i= 1 mod 3 by moving to (v,i +1). Notice that this edge is not available, 
and thus Player 2 is forced to move to t at times x = 2 mod 3. In partic- 
ular therefore, Player 1 wins from (v,0). The winning region for Player 1 is 
{(s, k), (t, k), (r, k), (u, 8k + 2), (v, 3k), (w, 38k +1) | ke N} 


The algorithmic question we consider is determining the set of vertices from 
which Player 1 wins starting at time 0. 
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3 Reachability Games 


We discuss a variant of temporal games that turns out to be central both for 
upper and lower bounds for solving games on temporal graphs. 

We call these punctual reachability games, which are played on a static graph 
and Player 1 aims to reach the target precisely at a target time. 


Definition 3. A punctual reachability game G = (V, E, so, F) is a game played 
on a static graph with vertices V = Vi © Vo, edges E C V?, an initial state so 
and set of target vertices F C V. An additional parameter is a target time T € N 
given in binary. Player 1 wins a play if and only if a vertex in F is reached at 
time T. 


Punctual reachability games are really just a reformulation of the membership 
problem for alternating finite automata (AFA) |f]]over a unary input alphabet. 
Player 1 wins the punctual reachability game with target T if, and only if, the 
word af is accepted by the AFA described by the game graph. Checking if a 
given unary word af is accepted by an AFA is complete for polynomial time if 
T is given in unary [20]} We first observe that it is PSPACE-hard if T is given in 
binary. We write in the terminology of punctual reachability games but the main 
argument is by reduction from the emptiness problem for unary AFA, which is 
PSPACE-compete [[8J[9]} We rely on the fact that the shortest word accepted 
by an AFA is at most exponential in the number of states. 


Lemma 1. Let G = (V, E, so, F) be a reachability game on a static graph. If 
there exist T € N so that Player 1 wins the punctual reachability game at target 
time T, then there exists some such T < 2V1. 


Proof. Assume towards contradiction that T > 2!"! is the smallest number such 
that Player 1 wins the punctual reachability game and consider some winning 
strategy ø. For any time k < T we can consider the set Sk C V of vertices 
occupied on any branch of length k on ø. By the pigeonhole principle, we observe 
k < k' < T with Sk = Sw, which allows to create a strategy o’ that follows o 
until time k, then continues (and wins) according to ø as if it had just seen 
a length k’ history leading to the same vertex. This shows that there exists a 
winning strategy for target time T — (k — k’), which contradicts the assumption. 


A lower bound for solving punctual reachability games is now immediate. 


Lemma 2. Solving punctual reachability games with target time T encoded in 
binary is PSPACE-hard. 


Proof. We reduce the non-emptiness problem of AFA over unary alphabets. In 
our terminology this is the decision problem if, for a given a reachability game 
G = (V, E, so, F) there exists some T € N so that Player 1 wins the punctual 
reachability game at target time T. This problem is PSPACE-complete [[8]} 
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By Lemmal|1| positive instances can be witnessed by a small target T < 2!V! 
and so we know that it is PSPACE-hard to determine the existence of such a 
small target time that allows Player 1 to win. 

Consider now the punctual reachability game G’ that extends G by a new 
initial vertex sọ that is owned by Player 1 and which has a self-loop as well as an 
edge to the original initial vertex sọ with target time T’ = 2!Y!. In G’, Player 1 
selects some number T < T” by waiting in the initial vertex for T” — T steps and 
then starts the game G with the target time T. Therefore, Player 1 wins in G” 
for target T” if, and only if, she wins for some T < 2!”! in G. 


Corollary 1. Solving reachability games on finite temporal graphs is PSPACE- 
hard. 


Proof. We reduce the punctual reachability game with target T to an ordinary 
reachability game on a finite temporal graph. This can be done by introducing 
a new vertex u as the only target vertex, so that it is only reachable via edges 
from vertices in F at time exactly T. That is E(s,u) = {T} and E(s,t) = [0,T 
for all s,t € V \ {u}. Now Player 1 wins the reachability game for target u if, and 


only if, she wins the punctual reachability game with target F at time T. 


A matching PSPACE upper bound for solving punctual reachability games, as 
well as reachability games on finite temporal graphs can be achieved by comput- 
ing the winning region backwards as follows For any game graph with vertices 
V=ViWVo, set S C V andi € {1,2}, let Pre;(S) C V denote the set of vertices 
from which Player 7 can force to reach S in one step. 


Pre;(S) = {v € V; | d(v,v’) € Ev’ € S}U {v € Via | V(u, 0’) € Ev! € 8} 


A straightforward induction on the duration T shows that Player i wins the 
punctual reachability game with target time T from vertex s if, and only if 
s € Pre; (F), the T-fold iteration of Pre; applied to the target set F. 

Notice that knowledge of Pre;(S) is sufficient to compute Pre*t!(S). By 
iteratively unfolding the definitiopof Pre*, we can compute Pre? (F) from 
Pre? (F) = F in polynomial space] Together with Lemma [2] we conclude the 


following. 


Theorem 1. Solving punctual reachability games with target time T encoded in 
binary is PSPACE-complete. 


1 For readers familiar with reachability games, the notion Prei(S) above is very similar 
to, but not the same as the k-step attractor of S: The former contains states from 
which Player 1 can force to see the target in exactly k steps, whereas the latter 
contains those where the target is reachable in k or fewer steps. 

2 To be precise, naively unfolding the definition requires O(T' +|V|*) time, exponential 
in (the binary encoded input) T, and O(|V|+log(T’)) space to memorise the current 
set Pre, C V as well as the time k < T in binary. 
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The same approach works for reachability games on finite temporal graphs if 
applied to the expansion up to horizon h(G), leading to the same time and space 
complexity upper bounds. The only difference is that computing Pref (F x {T}) 
requires to check edge availability at time T — k. 


Theorem 2. Solving reachability games on finite temporal graphs is PSPACE- 
complete. 


Proof. Consider a temporal game with vertices V=V,WV2, edges E : V? — 2N 
target vertices F C V and where T = h(G) is the latest time an edge is available. 
We want to check if starting in an initial state sọ at time 0, Player 1 can force 
to reach F at time T. In other words, for the game played on the expansion up 
to time T we want to decide if (so,0) is contained in Pre? (F x {T}). 

By definition of the expansion, we have Pre,(S x {n}) C V x {n—1} for all 
S CV and n < T. Since we can check the availability of an edge at time n in 
polynomial space, we can iteratively compute Pre? (Fx {T}) backwards, starting 
with Pre? (F x {T}) = Fx {T}, and only memorising the current iteration n < T 
and a set Wn C V representing Pre? (F x {T}) = Wn x {T — n}. 


4 Parity Games 


We consider Parity games played on periodic temporal graphs. As input we take 
a temporal graph G = (V, E) with period K, a partitioning V=V, WV. of the 
vertices, as well as a colouring col : V + C that associates a colour out of a 
finite set C C N of colours to every state. 

It will be convenient to write col(m) = max{col(s;) | 0 < i < k} for the max- 
imal colour of any vertex visited along a finite path 7 = (so, 0)(s1, 1)... (sx, k). 
The following relations RZ capture the guarantees provided by a strategy o if 
followed for one full period from vertex s. 


Definition 4. For a strategy o and vertex s € V define RZ C V x C be the 
relation containing (t,c) € RZ if, and only if, there exists a finite play t = 
(s,0)...(t, K) consistent with o, that starts in s at time 0, ends int at time K, 
and the maximum colour seen on the way is col(7) = c. We call RZ the summary 
of s with respect to strategy o. 

A relation BC V x C is s-realisable if there is a strategy o with B = RZ. 


Example 2. Consider the game in Fig. 2] where vertex u € V2 has colour 2 and 
all other vertices have colour 1. The graph has period K = 2. The relations 
{(t, 1)} and {(t, 2), (t’, 2)} are s-realisable, as witnessed by the strategies o(s) = t 
and o(s) = u), respectively. However, {(t,2)} is not s-realisable as no Player 1 
strategy guarantees to visit s then u then t. 


Lemma 3. Checking s-realisability is in PSPACE. That is, one can verify in 
polynomial space for a given temporal Parity game, state s € V and relation 
BCV xC whether B is s-realisable. 
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x =0O mod 2 


Fig. 2: The game from Example p] Labels on vertices and edges denote colours 
and available times, respectively. The graph has period 2. In two rounds, Player 1 
can force to end in ¢ having seen colour 1, or in either t or t/ but having seen a 
better colour 2. 


Proof. We reduce checking realisability to solving a reachability game on a 
temporal graph that is only polynomially larger. More precisely, given a game 
G = (V, E, col) consider the game G’ = (V’, E’, col’) over vertices V’ = V x C 
that keep track of the maximum colour seen so far. That is, the ownership 
of vertices and colours are lifted directly as (s,c) € Vi <= s€ Vı and 


col'(s,c) = col(s), and for any i € N, s,t,so € V, c,d € C, we let (t,d) 
be an i-successor of (s,c) if, and only if, both t is an i-successor of s and 
d = max{c, col(t)}. 

Consider some relation B C V x C. We have that B is s-realisable if, and only 
if, Player 1 wins the punctual reachability game on G’ from vertex (s, col(s)) at 
time 0, towards target vertices B C V’ at target time K. Indeed, any winning 
Player 1 strategy in this game witnesses that B is s-realisable and vice versa. By 
Theorem p] the existence of such a winning strategy can be verified in polynomial 
space by backwards-computing the winning region. 


The following defines a small, and PSPACE-verifiable certificate for Player 1 
to win the parity game on a periodic temporal graph. 


Definition 5 (Certificates). Given temporal parity game (V, E, col) with pe- 
riod K, a certificate for Player 1 winning the game from initial vertex so € V is 
a multigraph where the vertex set V' C V contains so, and edges E’ C V’xCxV' 
are labelled by colours, such that 


def 


1. For every s € V’, the set Post(s) = {(t,c) | (s,c,t) € E’} is s-realisable. 
2. The maximal colour on every cycle reachable from so is even. 


Notice that condition 1 implies that no vertex in a certificate is a deadlock. 
A certificate intuitively allows to derive Player 1 strategies based on those wit- 
nessing the realisability condition. 


Example 3. Consider the game from Example [1] played on the temporal graph 
with period 15. A certificate for Player 1 winning from state v at time 0 is 
depicted in Fig. |3| Indeed, the Player 1 strategy mentioned in Example fi] (aim 
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Fig. 3: A certificate that Player 1 wins the game in Example(1| from state v at 
time 0. 


to alternate between s and t) witnesses that Post(v) = {(s,3), (t,3), (r, 4)} is 
v-realisable because it allows Player 1 to enforce that after K = 15 steps from 
v, the game ends up in one of those states via paths whose colour is dominated 
by col(v) = 3 or col(r) = 4. 


Lemma 4. Player 1 wins the parity game on G from vertex so if, and only if, 
there exists a certificate. 


Proof. For the backward implication we argue that a certificate C allows to 
derive a winning strategy for Player 1 in the parity game G. By the realisability 
assumption (1), for each vertex s € V there must exist a Player 1 strategy os 
with RZs = Post(s) that tells her how to play in G for K rounds if the starting 
time is a multiple of K. Moreover, suppose she plays according to o, for K 
rounds and let ¢ and c be the vertex reached and maximal colour seen on the 
way. Then by definition of the summaries, (t,c) € RẸ» = Post(s) and so in the 
certificate C there must be some edge s —> t. 

Suppose Player 1 continues to play in G like this forever: From time i- K to 
(i+ 1)-K she plays according to some strategy c., determined by the vertex s; 
reached at time i- K. Any consistent infinite play p in G, chosen by her opponent, 
describes an infinite walk p’ in C such that the colour seen in any step i € N 
of p’ is precisely the dominant colour on p between rounds iK and (i + 1)K. 
Therefore the dominant colours seen infinitely often on p and p’ are the same 
and, by certificate condition (2) on the colouring of cycles, even. We conclude 
that the constructed strategy for Player 1 is winning. 

For the forward implication, assume that Player 1 wins the game on G from 
vertex s at time 0. Since the game G is played on a temporal graph with period 
K, its expansion up to time K — 1 is an ordinary parity game on a static graph 
with vertices V x {0,1,..., K—1} where the second component indicates the time 
modulo K. Therefore, by positional determinacy of parity games (Proposition|1), 
we can assume that Player 1 wins in G using a strategy o that is itself periodic. 
That is, o(hv) = o(h'v) for any two histories h, h’ of lengths |h| = |h'| mod K. 
Moreover, we can safely assume that o is uniform, meaning that it is winning 
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from any vertex (s, 0) for which a winning strategy exists. Such a strategy induces 
a multigraph C = (V, E’) where the edge relation is defined by (s,c,t) € EB’ <== 
(t,c) € RZ. It remains to show the second condition for C to be a certificate, 
namely that any cycle in C, reachable from the initial vertex so, has an even 
maximal colour. Suppose otherwise, that C contains a reachable cycle whose 
maximal colour is odd. Then there must be play in G that is consistent with o 
and which sees the same (odd) colour infinitely often. But this contradicts the 
assumption that ø was winning in G in the first place. 


Our main theorem is now an easy consequence of the existence of small 
certificates. 


Theorem 3. Solving parity games on periodic temporal graphs is PSPACE- 
complete. 

Proof. Hardness already holds for reachability games Lemma |2| For the up- 
per bound we show membership in NPSPACE and use Savitch’s theorem. By 
Lemma f] it suffices to guess and verify a candidate certificate C. These are by 
definition polynomial in the number of vertices and colours in the given temporal 
parity game. Verifying the cycle condition (2) is trivial in polynomial time and 
verifying the realisability condition (1) is in PSPACE by Lemma B] 


Remark 1. The PSPACE upper bound in Theorem B]can easily be extended to 
games on temporal graphs that are ultimately periodic, meaning that there exist 
T,K € N so that for all n > T, s—>t implies sE t Such games can be 
solved by first considering the periodic suffix according to T heorem [8] thereby 
computing the winning region for Player 1 at time exactly T, and then solving 
the temporal reachability game with horizon T. 


5 Monotonicity 


In this section, we consider the effects of monotonicity assumptions on the edge 
relation with respect to time on the complexity of solving reachability games. We 
first show that reachability games remain PSPACE-hard even if the edge relation 
is decreasing (or increasing) with time. We then give a fragment for which the 
problem becomes solvable in polynomial time. 


Increasing and Decreasing temporal graphs: Let the edge between vertices 

u,v € V of a temporal graph be referred to as decreasing if uty implies 
u — v for all i € N, i.e. edges can only disappear over time. Similarly, call the 
edge increasing if for all i € N we have that u — v implies u kam v; i.e. an edge 
available at current time continues to be available in the future. A temporal 
graph is decreasing (increasing) if all its edges are. We assume that the times 
at which edge availability changes are given in binary. More specifically, every 
edge is given as inequality constraint of the form By w(x) = x < n (respectively 
x > n) for some n E N. 
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Fig. 4: Reduction from a punctual reachability game to a reachability game on a 
temporal graph that is finite and decreasing, see Theorem g] Components added 
are shown in red. 


Although both restrictions imply that the graph is ultimately static, we ob- 
serve that solving reachability games on such monotonically increasing or de- 
creasing temporal graphs remains PSPACE-complete. 


Theorem 4. Solving reachability and Parity games on decreasing (respectively 
increasing) temporal graphs is PSPACE-complete. 


Proof. The upper bound holds for parity games as the description of the tempo- 
ral graph explicitly includes a maximal time T from which the graph becomes 
static. One can therefore solve the Parity game for the static suffix graph (in 
NP) and then apply the PSPACE procedure (Theorem pj to solve for temporal 
reachability towards the winning region at time T. Alternatively, the same upper 
bound also follows from Theorem Bland Remark [1] 

For the lower bound we reduce from punctual reachability games which are 
PSPACE-hard by Lemma[2] Consider a (static) graph G and a target time T € N 
given in binary. Without loss of generality, assume that the target vertex v has no 
outgoing edges. We convert G into a temporal graph G’ with V’ = VU{w, T, L}, 
Vi = (Vi\{v})U{w}, Vy = V’\ Vý and new target T. The vertex L is a sink state 
and the original target vertex v is now controlled by Player 2. Edge availabilities 
are v—> L ifr <T-1, vw ifs <T +1, wT ifx <T +1, and all 
other edges disappear after time T +1. The constructed temporal graph is finite 
and decreasing. See Fig. 4] The construction ensures that the only way to reach 
T is to reach v at time T, w at time T + 1 and take the edge from w to T at 
time T + 1. Player 1 wins in G” if and only if she wins the punctual reachability 
game on G. 

A similar reduction works in the case of increasing temporal graphs by switch- 
ing the ownership of vertices v and w. The vertex v, now controlled by Player 1 
has the edge v = w at times z > T and the edge v—> L at all times. The 
vertex w now controlled by Player 2 has the edge w —> T available at all times 
but the edge w > L becomes available at time x > T +2. oO 
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Declining and improving temporal games: We now consider the restriction where 
all edges controlled by one player are increasing and those of the over player are 
decreasing. Taking the perspective of the system Player 1, we call a game on 
a temporal graph declining if all edges u—> v with u € V are decreasing and 
all edges u —» v with u € Vz are increasing. Note that declining is a property 
of the game and not the graph as the definition requires a distinction based on 
ownership of vertices, which is specified by the game and not the underlying 
graph. From now on, we refer to such games as declining temporal reachability 
(or parity) games. Notice that Player 1 has fewer, and Player 2 has more choices 
to move at later times. Analogously, call the game improving if the conditions are 
reversed, i.e., all edges u—> v with u € Vj are increasing and all edges u — v 
with u € V2 are decreasing. 

We show that declining (and improving) temporal reachability games can be 
solved in polynomial time. 


Theorem 5. Solving declining (respectively improving) temporal reachability 
games is in P. 


Proof. We first give the proof for declining games. Consider the reachability 
game on the expansion with vertices V x N such that the target set is F x N. 
For k € N let Wk C V be the set of those vertices u such that Player 1 has a 
winning strategy from (u, k). We first show that 


Wis C W; (1) 


For sake of contradiction, suppose there exists u € W;41 \ W;. Let o} 41 bea 
(positional) winning strategy from (u,i+1) for Player 1 in the expansion. Since 
u Z Wi, by positional determinacy of reachability games (Proposition|]) , Player 2 
has a winning strategy o? from (u, i). Consider a strategy o} for Player 1, such 
that for all v € Vi, o}(v,k) = al (v,k + 1), for all k > i. Similarly, let o?,, 
be the strategy for Player 2, such that for all v € V2, 07,,(v,k + 1) = o? (v, k), 
for all k > i, Note that this is well defined because by definition of declining 

: k+l cea k Th oo sa adi k+l 
games, i.e, v —> u implies v —> u for all v € Vi, and v —> u implies v —> u, for 
all v € Vz. Starting from the vertex (u,i+ 1), the pair of strategies (oj, 1, 07,4) 
defines a unique play 7;41, which is winning for Player 1. Similarly, the pair of 
strategies (o},07) define a play m; which is winning for Player 2 starting from 
(u,7). However, the two plays visit the same set of states, particularly, (v, k) is 
visited in 7; if and only if (v, k + 1) is visited in 7,41. Therefore, either both are 
winning for Player 1 or both are losing for Player 2, which is a contradiction. 
Let N CN be the set of times at which the graph changes, i.e. 


N = {c | Iua (£) = x <c, where <4 € {<,>}}} 


Let m © max(N) be the latest time any edge availability changes. We show that 
Wm = Wp for all k > m. To see this, note that Wm is equal to the winning 
region for Player 1 in the (static) reachability game played on Gm = (V, Em), 
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Algorithm 1 Algorithm for declining games with set of change times N and 
m = max(N) 


W + Solve(Gm) > Computes Player 1 winning region in Gm 
while N Æ ý do 
n + maz(N) 
if (Preı(W x {n}) =W then 
N¢N\n > Accelerate to next change time 
else 
W + Pre\(W) 
N+} NU{n-1}\ {n} 
end if 
end while 


where Em = {(u,v) | u-v}. Consider a (positional) winning strategy Om 
for Player 1 in Gm and define a positional strategy o (v, k) = om(v), for k > m. 
Since the graph is static after time m, this is well defined. Starting from a vertex 
(u, k), a vertex (v, k +k’) is visited on a ø-consistent path if and only if there is a 
Om-consistent path u — „ v. Therefore, ø is a winning strategy from any vertex 
(v, k) such that k > m and v € Wm. Moreover, the set Wm can be computed in 
time O(|V|?) by solving the reachability game on Gm [13, Theorem 12]. 

To solve reachability on declining temporal games, we can first compute the 
winning region Wm in the stabilised game Gm. This means Wm x [m,co) is 
winning for Player 1. To win the declining temporal reachability game, Player 1 
can play the punctual reachability game with target set Wm at target time 
m. The winning region for Player 1 at time 0 can therefore be computed as 
Pre? (Wm x {m}) as outlined in the proof of Theorem |2} Note that naively this 
only gives a PSPACE upper bound as in the worst case, we would compute Pre, 
an exponential (m) times. 

To overcome this, note that in the expansion graph Prei (Wm x {m}) = 
Wm-—i X {m — i}. According to Eq. (1), Wm-i C Wmv for i’ > i. Let 7,7’ be 
such that m—i and m—7’ are both consecutive change points, i.e, m—i,m—i’ € N 
and YVL € N.L < m—i' V£>m-—i. Since the edge availability of the graph does 
not change between time m — 2’ and m — i, we have W,,_;-1 = Wm-—i implies 
Wm—iv = Wm_i- Therefore, we can accelerate the Pre; computation and directly 
move to the time step m— i’, i.e, the i'th iteration in the computation. This case 
is illustrated at time n’ = m — 7’ in Fig. 

With this change, our algorithm runs the Pre; computation at most |V|+|N, 
as each Pre, computation either corresponds to a step a time in N when the 
graph changes, or a step in which the winning region grows such as at time n in 
Fig. [5| Since each Pre; computation can be done in polynomial time, we get a 
PTIME algorithm in this case, shown in Algorithm 

The case for improving temporal reachability games can be solved similarly. 
Instead of computing the winning region for Player 1 in Gm, we start with 
computing the winning region W2 for Player 2 in Gm and switch the roles of 
Player 1 and Player 2, i.e, Player 2 has the punctual reachability objective with 
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n! n m 


Fig. 5: Illustration of Algorithm[]]. The blue vertices at time 7 denote the winning 
region W; for Player 1. The times n,n’ € N and Pre, computation at change 
point n increases the winning region but is stable at time m. 


target set W,? and target time m, which can be solved as above. This gives us 
an algorithm to compute the winning region for Player 2 and by determinacy 
of reachability games on infinite graphs, we can compute the winning region for 
Player 1 at time 0 as well. 


Remark 2. Algorithm iL] also works for parity objectives by changing step 1, 
where Solve(Gm) would amount to solving the parity game on the static graph 
Gm. This can be done in quasi-polynomial time and therefore gives a quasi- 
polynomial time algorithm to solve declining (improving) temporal parity games 
and in particular, gives membership in the complexity class NP N coNP. 


Since the declining (improving) restriction on games on temporal graphs 
allow for improved algorithms, a natural question is to try to lift this approach 
to a larger class of games on temporal graphs. Note that the above restrictions 
are a special case of eventually periodic temporal graphs with a prefix of time m 
followed by a periodic graph with period 1. Now, we consider temporal graphs 
of period K > 1 such that the game arena is declining (improving) within 
each period. Formally, a game on a temporal graph G is periodically declining 
(improving) if there exists a period K such that for all k € N, k € E(u, v) if and 
only if k+K € E(u,v); and the game on the finite temporal graph resulting from 
G by making the graph constant from time K onwards, is declining (improving). 
We prove that this case is PSPACE-hard, even with reachability objectives. 


Theorem 6. Solving periodically declining (improving) temporal reachability games 
is PSPACE-complete. 


Proof. The upper bound follows from the general case of parity games on periodic 
temporal graphs in T heorem|3} The lower bound is by reduction from punctual 
reachability games. See Fig. [6 Given a (static) graph G with target state v and 
target time T, we obtain a periodically declining game G” with period K = T+1, 
vertices VU{w, L, T}, new target T, such that V/ = ViU{w, L, T} and V3 = V2. 
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Fig. 6: Reduction from a punctual reachability game to a reachability game on a 
temporal graphs that is periodic and declining, see Theorem [6} Parts added are 
shown in red. 


We assume without loss of generality that the original target v is a Player 1 
vertex, i.e, v € Vi. 


We describe the edge availability in G” up to the period K = T + 1. For all 
edges (s, t) of the original graph G, such that s € V1, the edge s > t is available 
if and only if x < T. Moreover for all s € V; \ {v}, there is a new edge s > L 
available at all times x < T. For all s € V2, there is an edge s —) tis available at 
all times (until end of period) and s—*+ L is available after time x > T. These 
edges ensure that if a play in the original punctual reachability game ends in a 
vertex of the game other than v at time T, then Player 2 can force the play to 
reach the sink state L and win. 


From the original target v, there is an edge to the new state w at all times. 
From the state w, there are edges w — L at all times and w—>T if z = 0. If 
the state w is reached at time k such that 1 < k < T + 1, then the play is forced 
to go to L. The only winning strategy for Player 1 is to reach v at time T, w 


at time T + 1 at which the time is reset due to periodicity. The edge w TH T is 
now available for Player 1 and they can reach the new target T. 


The lower bound for the case of periodically increasing temporal reachability 
games follows by the same construction and using the duality between improving 
and declining games on temporal graphs. Given a punctual reachability game 
G with vertices V = Vi © V2 with target set F, we obtain the dual punctual 
reachability game G with same target time by first switch the ownership of 
vertices, i.e, V; = V3—i, i € {1,2} and make the new target as V \ F. It is easy 
to see that Player 1 wins G if and only if Player 2 wins G. 


Applying the same construction as shown in Fig. [el to G, we obtain a pe- 
riodically declining temporal reachability game GC’, preserving the winner. Now 
switching the ownership of vertices in G yields a periodically improving tem- 
poral reachability game G” which is winning for Player 1 if and only if Player 1 
wins G. 
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6 Conclusion 


In this work we showed that parity games on ultimately periodic temporal graphs 
are solvable in polynomial space. The lower bound already holds for the very 
special case of punctual reachability games, and the PSPACE upper bound, which 
improves on the naive exponential-space algorithm on the unfolded graph, is 
achieved by proving the existence of small, PSPACE-verifiable certificates. 

We stress again that all constructions are effective no matter how the tempo- 
ral graphs are defined, as long as checking edge availability for binary encoded 
times is no obstacle. In the paper we use edge constraints given in the existential 
fragment of Presburger arithmetic but alternate representations, for example us- 
ing compressed binary strings of length h(G) given as Straight-Line Programs [5] 
Section 3] would equally work. Checking existence of edge at time i would cor- 
respond to querying whether the it” bit is 1 or not which is P-complete [27, 
Theorem 1]. 

The games considered here are somewhat orthogonal to parity games played 
on the configuration graphs of timed automata, where time is continuous, and 
constraints are quantifier-free formulae involving possibly more than one variable 
(clocks). Solving parity games on timed automata with two clocks is complete 
for EXP but is in P if there is at most one one clock [16] Contribution 3(d)]. 
Games on temporal graphs with quantifier-free constraints corresponds to a sub- 
class of timed automata games with two-clocks, with intermediate complexity 
of PSPACE. This is because translating a temporal graph game to a timed au- 
tomata game requires two clocks: one to hold the global time used to check the 
edge predicate and one to ensure that time progresses one unit per step. 

An interesting continuation of the work presented here would be to consider 
mean-payoff games [II] played on temporal graphs, possibly with dynamic step- 
rewards depending on the time. If rewards are constant but the edge availability 
is dynamic, then our arguments for improved algorithms on declining/improving 
graphs would easily transfer. However, the PSPACE upper bound using sum- 
maries seems trickier, particularly checking realisability of suitable certificates. 


Acknowledgements This work was supported by the Engineering and Physical 
Sciences Research Council (EPSRC), grant EP/V025848/1. We thank Viktor 
Zamaraev and Sven Schewe for fruitful discussions and constructive feedback. 


References 


1. Automata Logics, and Infinite Games: A Guide to Current Research. Springer- 
Verlag (2002), netpa://d0s.org/10.1007/3-640-36367-4 | 

2. Akrida, E.C., Mertzios, G.B., Spirakis, P.G., Zamaraev, V.: Temporal vertex cover 
with a sliding time window. Journal of ipte and System Sciences 107, 108- 
123 (2020). : i.org : i. i j. jcss.2019.08.002 

3. Alur, R., Dill, D.L.: A theory of timed automata. Theor. Comput. Sci. 126(2), 183 


— 235 (1994). https: //doi.org/10.1016/0304-3975 (94) 90010-8 


96 P. Austin, S. Bose, and P. Totzke 


4. Avni, G., Ghorpade, P., Guha, S.: A Game of Pawns. In: International Conference 
on Concurrency Theory. Leibniz International Proceedings in Informatics (LIPIcs), 
vol. 279, pp. 16:1-16:17. Schloss Dagstuhl — Leibniz-Zentrum fiir Informatik (2023). 

5. Babai, L., Szemeredi, E.: On the complexity of matrix group problems i. In: Annual 
Symposium on Foundations of Computer Science. pp. 229-240 (1984). 

6. Calude, C.S., Jain, S., Khoussainov, B., Li, W., Stephan, F.: Deciding parity games 
in Sigg aber time. In: Aste ea on i of Computing. pp. 252-263 
(2017). 

7. Chandra, A.K., ec D.C. me; i ernatio 
(JACM) 28(1 i 114-133 (1981) (mt 

8. Chatterjee, K., Henzinger, T.A., P u, V. y : Complexity 
and Robustness. Logical Methods in Computer Science Volume 7, Issue 4 (Dec 
inl mt te 

9. Colcombet, T, Fijalkow, N.: Universal T and Good for Games Automata: 
New Tools for Infinite Duration Games. In: International Conference on Founda- 
tions of Software Science and Computational Structures. LNCS, vol. 11425, pp. 
1-26. Springer (2019). https ://doi.org/10.1007/978-3-030-17127-8_1 

10. De Carufel, J.L., Flocchini, P., Santoro, N., Simard, F.: Cops & robber on periodic 
temporal graphs: Characterization and improved bounds. In: Structural Informa- 
tion and Communication Seia pp. 386-405. Sanaa Nature Switzerland 


(2023) [https://doi .org/10. 1007/978-3-031-32733-9_17 | 
11. Ehrenfeucht, A., ivcels i, J.: Positiona AEEA or mean payoff games. nter- 
national Journal of Game Theory 8(2), 109-113 (Jun 1979). h 


12. Erlebach, T., Hoffmann, M., Kammer, F.: On temporal graph exploration. Journal 
of Com puter and S stem Sciences 119, 1-18 (2021). https://doi.org/https: 


13. 


ijalkow, N., Bertrand, 5 Bouyer-Decitre, P., Brenguier, R., Carayol, A., Fearn- 
ley, J., Gimbert, H., Horn, F., Ibsen-Jensen, R., Markey, N., Moninege, B. “Novotný, 
P., Randour, M. Sankur O., Schmitz, S. „Serre O., Skomra, M.: Games on graphs 
(2023). https://doi.org/10.48550/arXiv.2305. 10546 | 

14. Flocchini, P., o B., ET N.: Exploration of periodically varying graphs. 

In: Algorithms and Computation. pp. 534-543. Springer Berlin Heidelberg (2009). 


15. Haase, C.: A survival guide to presburg 


16. Hansen, TD, Tbsen- ence R., Miltersen, P.B.: A fa algorithm fo 
solving one- clok priced timed games (2013). 

17. Holme, P., Saramäki, J.: Temporal Network Theory (01 2019)| https: //doi.org/| 

18. Holzer, M.: On emptiness and counting for alternating finite automata. In: Inter- 


national Conference on Developments in Language Theory. pp. 88-97 arcane 
19. Janéar, P., Sawa, Z.: A note on emptiness for alternating finite_a 


om A 
one-letter alghahst. Inf. Process. Lett. 104(5), 164-167 (2007). https: ree oT 


20. Jiang, T., Ravikumar, B.: A note on the space complexity of some decision problems 


for finite automata. Inf. Process. Lett. 40(1), 25-31 (1991). |https://doi.org/ 


https: //doi.org/10.1016/S0020-0190 (05) 80006-7 


Parity Games on Temporal Graphs 97 


21. Jurdzifiski, M., Trivedi, A.: Reachability-time games on timed automata. 
In: International Colloquium on Automata, Languages and Programming. 


pp. 838-849. Springer Berlin Heidelberg (2007). | https://doi.org/10.1007/ 


22. Jurdzinski, M.: Deciding the winner in parity games is in up N co-up. Inf. Pro- 
cess. Lett. 68(3), 119-124 (1998). 

23. Jurdziński, M., Lazić, R.: Succinct Progress Measures for Solving Parity Games. In: 
Annual IEEE Symposium on Logic in Computer Science. pp. 1-9. IEEE Computer 
Society (2017). https: //doi.org/10.1109/LICS. 2017. 8005092 | 

24. Kuhn, F., Lynch, N., Oshman, R.: Distributed computation in dynamic networks. 
In: Symposium on Theory of Computing. p. 513-522. STOC ’10, Association for 
Computing Machinery (2010) 

25. Lehtinen, K., Parys, P., Schewe, S., V D.: A R i i 
Parity Games in Quasi 


polynomial Time. Logi hods i D 
18(1), 8:1-18 (2022). https://doi.org/10.46298/1mcs-18(1:8) 2022 


26. 


27. Lifshits, Y., Lohrey, M.: Querying and embedding compressed texts. In: Interna- 
tional Symposium on Mathematical Foundations of Computer Science. pp. 681- 
692, Springer Berlin Heidelberg (2006). [netps:: //doi .org/10.1007/11821069_59 | 

28. Maler, O., Pnueli, A., Sifakis, J.: On the synthesis of discrete controllers for timed 
systems. In: International Symposium on Theoretical Aspects of Computer Sci- 


ence. pp. 229-242. Springer Berlin Heidelberg (1995). https: //doi.org/10.1007/ 


29. Mertzios, G.B., Molter, H., Niedermeier, R., Zamaraev, V., Zschoche, P.: Com- 
puting maximum matchings in temporal graphs. Journal of Computer and System 
Sciences 137, 1-19 (2023)! https://doi.org/https://doi.org/10.1016/j.jcss. 


30. Mertzios, G.B., Molter, H., Zamaraev, V.: Sliding window temporal graph coloring. 
Journal of Computer and System Sciences 120, 97-115 (2021). https://doi org 


31. Michail, O.: An Introduction to Temporal Graphs: An Algorithmic Perspective, pp. 


308-343. Springer International Publishing (2015). https://doi.org/10.1007/ 


32. Michail, O., Chatzigiannakis, I., Spirakis, P.G.: Causality, influence, and computa- 
tion in possibly disconnected synchronous dynamic networks. Journal of Parallel 
and Distributed Computing 74(1), 2016-2026 (2014).[https://doi .org/10.1016/ | 

33. Michail, O., Spirakis, P.G.: Traveling salesman problems in temporal graphs. In: 
International Symposium on Mathematical Foundations of Computer Science. pp. 
553-564. Springer Berlin Heidelberg (2014). 

34. Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: Annual Sympo- 
sium on Principles of Programming Languages. p. 179-190. POPL ’89, Association 
for Computing Machinery (1989). 

35. Pnueli, A.: The temporal logic of programs. In: Annual Symposium on Foundations 
of Computer Science. p. 46-57. SFCS ’77, IEEE Computer Society (1977). 


/doi.org/10.1109/SFCS.1977.32 


98 P. Austin, S. Bose, and P. Totzke 


36. Ravi, R.: Rapid rumor ramification: Approximating the minimum broadcast time. 
In: Proceedings 35th Annual Symposium on Foundations of Computer Science. pp. 
202-213 (1904), heps://de..org/10.1109/SFCS. 1994. 365693) 

37. Scarpellini, B.: Complexity of subcases of presburger arithmetic. Transactions of 
the American Mathematical Society 284, 203-218 (1984). /nttps://doi.org/10.| 

38 


. Trivedi, A.: Competitive optimisation on timed automata. Ph.D. thesis, University 
of Warwick (April 2009) 


Open Access This chapter is licensed under the terms of the Creative Commons 
Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), 
which permits use, sharing, adaptation, distribution and reproduction in any medium 
or format, as long as you give appropriate credit to the original author(s) and the 
source, provide a link to the Creative Commons license and indicate if changes were 
made. 

The images or other third party material in this chapter are included in the 
chapter’s Creative Commons license, unless indicated otherwise in a credit line to the 
material. If material is not included in the chapter’s Creative Commons license and 
your intended use is not permitted by statutory regulation or exceeds the permitted 
use, you will need to obtain permission directly from the copyright holder. 


Categorical Semantics 


Check for 
updates 


Drawing from an Urn is Isometric 


Bart Jacobs 

Institute for Computing and Information Sciences, 

Radboud University, Nijmegen, The Netherlands 
bart@cs.ru.nl 


Abstract. Drawing (a multiset of) coloured balls from an urn is one of the most 
basic models in discrete probability theory. Three modes of drawing are com- 
monly distinguished: multinomial (draw-replace), hypergeometric (draw-delete), 
and Pólya (draw-add). These drawing operations are represented as maps from 
urns to distributions over multisets of draws. The set of urns is a metric space 
via the Wasserstein distance. The set of distributions over draws is also a metric 
space, using Wasserstein-over-Wasserstein. The main result of this paper is that 
the three draw operations are all isometries, that is, they preserve the Wasserstein 
distances. 


Keywords: probability - urn drawing - Wasserstein distance. 


1 Introduction 


We start with an illustration of the topic of this paper. We consider a situation with a set 
C = {R,G, B} of three colours: red, green, blue. Assume that we have two urns v1, v2 
with 10 coloured balls each. We describe these urns as multisets of the form: 


vı = 8/G)+2|B) and — v= 5 R) +. 4|G) +1). 


Recall that a multiset is like a set, except that elements may occur multiple times. Here 
we describe urns as multisets using ‘ket’ notation | — ). It separates multiplicities of 
elements (before the ket) from the elements in the multiset (inside the ket). Thus, urn 
vı contains 8 green balls and 2 blue balls (and no red ones). Similarly, urn v2 contains 
5 red, 4 green, and 1 blue ball(s). 

Below, we shall describe the Wasserstein distance between multisets (of the same 
size). How this works does not matter for now; we simply posit that the Wasserstein dis- 
tance d(v1, v2) between these two urns is 5 — where we assume the discrete distance 
on the set C of colours. 

We turn to draws from these two urns, in this introductory example of size two. 
These draws are also described as multisets, with elements from the set C = {R, G, B} 
of colours. There are six multisets (draws) of size 2, namely: 


2|R) 1)R)+1]G) 2/G) 1/R)+1/B) 2|B) 1/G)4+1|B). dd 


As we see, there are three draws with 2 balls of the same colour, and three draws with 
balls of different colours. 
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We consider the hypergeometric probabilities associated with these draws, from the 
two urns. Let’s illustrate this for the draw 1|G) + 1| B} of one green ball and one blue 
ball from the urn vı. The probability of drawing 1|G) + 1| B) is 38; it is obtained as 
sum of: 


- nest drawing-and-deleting a green ball from vı = 8|G) + 2|B), with probability 
jo: It leaves an urn 7|G) + 2| B}, from which we can draw a blue ball van pe 
bility 2 2, Thus drawing “first green then blue” happens with probability {t Y ay 2 =Ż. 

- Similarly, the probability of drawing “first blue then green” is 2 io z- = 5 
We can similarly compute the probabilities for each of the above six draws (1) from urn 
v1. This gives the hypergeometric distribution, which we write using kets-over-kets as: 


hg[2|(v1) = 28 s |21G)) +3 18) 11G) +11B)) +4 $ |21B)). 


The fraction written before a big ket is the probability of drawing the multiset (of size 
2), written inside that big ket, from the urn v1. 

Drawing from the second urn v2 gives a different distribution over these multi- 
sets (ip. Since urn v2 contains red balls, they additionally appear in the draws. 


hg[2\(v2) = 2|21R)) + 441 R) +1|G)) +4 2 |216) ) 
tair) +118))+ 4 hic) +1|B)). 


We can also compute the distance between these two hypergeometric distributions over 
multisets. It involves a Wasserstein distance, over the space of multisets (of size 2) 
with their own Wasserstein distance. Again, details of the calculation are skipped at this 
stage. The distance between the above two hypergeometric draw-distributions is: 


d(hg[2\(v1), hgl2\(v2)) = 4 = d(vr, v2). 


This coincidence of distances is non-trivial. It holds, in general, for arbitrary urns (of the 
same size) over arbitrary metric spaces of colours, for draws of arbitrary sizes. More- 
over, the same coincidence of distances holds for the multinomial and Pólya modes of 
drawing. These coincidences are the main result of this paper, see Theorems [I] [2] and[] 
below. 

In order to formulate and obtain these results, we describe multinomial, hypergeo- 
metric and Pólya distributions in the form of (Kleisli) maps: 


mn hg[K] 
p(x) EL, p(mM[K](Xx)) Minx) (2) 


They all produce distributions (indicated by D), in the middle of this diagram, on mul- 
tisets (draws) of size K, indicated by M[K], over a set X of colours. Details will be 
provided below. Using the maps in (2), the coincidence of distances that we saw above 
can be described as a preservation property, in terms of distance preserving maps — 
called isometries. At this stage we wish to emphasise that the representation of these 
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different drawing operations as maps in has a categorical background. It makes it 
possible to formulate and prove basic properties of drawing from an urn, such as natu- 
rality in the set X of colours. Also, as shown in [8] for the multinomial and hypergeo- 
metric case, drawing forms a monoidal transformation (with ‘zipping’ for multisets as 
coherence map). This paper demonstrates that the three draw maps (2) are even more 
well-behaved: they are all isometries, that is, they preserve Wasserstein distances. This 
is anew and amazing fact. 

This paper concentrates on the mathematics behind these isometry results, and not 
on interpretations or applications. We do like to refer to interpretations in machine learn- 
ing where the distance that we consider on colours in an urn is called the ground 
distance. Actual distances between colours are used there, based on experiments in 
psychophysics, using perceived differences [16]. 

The Wasserstein — or Wasserstein-Kantorovich, or Monge-Kantorovich — dis- 
tance is the standard distance on distributions and on multisets, going back to [12]. 
After some preliminaries on multisets and distributions, and on distances in general, 
Sections [4] and [5] of this paper recall the Wasserstein distance on distributions and on 
multisets, together with some basic results. The three subsequent Sections|6]-[8]demon- 
strate that multinomial, hypergeometric and Pólya drawing are all isometric. Distances 
occur on multiple levels: on colours, on urns (as multisets or distributions) and on draw- 
distributions. This may be confusing, but many illustrations are included. 


2 Preliminaries on multisets and distributions 


A multiset over a set X is a finite formal sum of the form $; n;|x:}, for elements 
x; E€ X and natural numbers n; € N describing the multiplicities of these elements 
xi. We shall write M(X) for the set of such multisets over X. A multiset p E€ M(X) 
may equivalently be described in functional form, as a function y: X — N with finite 
support: supp(y) == {x € X | y(x) Æ 0}. Such a function y: X — N can be 
written in ket form as ’ ` ex y(x)|x). We switch back-and-forth between the ket and 
functional form and use the formulation that best suits a particular situation. 

For a multiset ¢ € M(X) we write ||y|| € N for the size of the multiset. It is the 
total number of elements, including multiplicities: 


lel = So y). 


cTEX 


For a number K € N we write M[K](X) C M(X) for the subset of multisets of size 
K. There are ‘accumulation’ maps acc: X — M[K](X) turning lists into multisets 
via acc (£1,... £K) = 1]"1)+---+1|xx). For instance acc (c, b, a,c, a,c) = 2|a)+ 
1|b) +3]c). A standard result (see [10]) is that for a multiset y € M[K](X) there are 


(vy) = at many sequences x € X* with acc(x) = p, where y] = Į [, y(z)!. 


Multisets y, € M(X) can be added and compared elementwise, so that (p + 
Y) (x) = v(x) +(x) and p < p means y(x) < y(x) for all x € X. In the latter case, 
when y < y, we can also subtract Y — p elementwise. 
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The mapping X > M(X) is functorial: for a function f: X — Y we have 
M(f): M(X) > M(Y) given by M(f)()() = Dove pty) 2(2). This map M( f) 
preserves sums and size. 

For a multiset r € M(X x Y) on a product set we can take its two marginals 
M(m1)(7) E€ M(X) and M(72)(r) E€ M(Y) via functoriality, using the two projec- 
tion functions 71: X x Y > X and m2: X x Y > Y. Starting from y € M(X) and 
we M(Y), we say that r € M(X x Y) is a coupling of 9,4 if y and w are the two 
marginals of 7. We define the decoupling map: 


M(X xY) depl := (M (m1), M(T2)}) 


M(X) x M(Y) (3) 


The inverse image dep!~' (p, Y) C M(X x Y) is thus the subset of couplings of y, y. 


A distribution is a finite formal sum of the form )°,7r;|x;) with multiplicities 
ri € [0,1] satisfying }°, r; = 1. Such a distribution can equivalently be described as a 
function w: X — [0,1] with finite support, satisfying 5°, w(a) = 1. We write D(X) 
for the set of distributions on X. This D is functorial, in the same way as M. Both D and 
M are monads on the category Sets of sets and functions, but we only use this for D. 
The unit and multiplication / flatten maps unit: X + D(X) and flat: D?(X) > D(X) 
are given by: 


unit(x) := 1|2) Atl) =X X QW)-w(x)] |x). A 


xEX \wED(x) 


Kleisli maps c: X — D(Y) are also called channels and written as c: X -> Y. Kleisli 
extension c >= (—): D(X) + D(Y) for such a channel, is defined on w € D(X) as: 


cw := flat(D(c)(w)) = 5 (= w(x): sow) |y). 


yEY \zEX 


Channels c: X ~» Y and d: Y > Z can be composed to do c: X => Z via (do 
c)(x) := d >= c(x). Each function f: X — Y gives rise to a deterministic channel 
<f> = unit o f: X ~» Y, that is, via < f>(x) = 1| f(<)}. 

An example of a channel is arrangement arr: M[K](X) — D(X*). It maps a 
multiset p E€ M[K](X) to the uniform distribution of sequences that accumulate to y. 


arl) = Y Gayl) = 5 2 


aw€acc—1(y) w€acc—1() 


One can show that <acc» © arr = D(acc) o arr = unit: M[K](X) > D(M[K](X)). 
The composite in the other direction produces the uniform distribution of all permuta- 
tions of a sequence: 


1 
arr © <acc>) = arr o acc = prm where prm(ax) := 5 Fal |t(x)), (6) 


t: KK 
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in which t(z1, noe zp) = (@4(1),---,24(K)). In writing t: K 3 K we implicitly 
identify the number K with the set {1,..., K}. 

Each multiset p € M(X) of non-zero size can be turned into a distribution via 
normalisation. This operation is called frequentist learning, since it involves learning a 
distribution from a multiset of data, via counting. Explicitly: 


Flrn(p) := -> H hae ps 


For instance, if we learn from an urn with three red, two green and five blue balls, we 
get the probability distribution for drawing a ball of a particular colour from the urn: 


Firn (3| R) + 2|G) +5] B)) = lR) + lG) + 5|B). 


This map Firn is a natural transformation (but not a map of monads). 
Given two distributions w € D(X) and p € D(Y), we can form their parallel 
product w & p € D(X x Y), given in functional form as: 


(w8 p)(x,y) = w(x) - ply). 


Like for multisets, we call a joint distribution 7 € D(X x Y) a coupling of w € D(X 
and p € D(Y) if w, p are the two marginals of 7, that is if, D(7)(7) = w and D(m2) = 
p. We can express this also via a decouple map dep! = (D(71),D(m2)) as in (8). 


An observation on a set X is a function of the form p: X — R. Such a map p, 
together with a distribution w € D(X), is called a random variable — but confusingly, 
the distribution is often left implicit. The map p: X — R will be called a factor if it 
restricts to non-negative reals X —> Ryo. Each element x € X gives rise to a point 
observation 1,: X — R, with 1,(2’) = 1 if x = v’ and 1,(x') = 0 if x # x’. Fora 
distribution w € D(X) and an observation p: X — R on the same set X we write w = 
p for the validity (expected value) of p in w, defined as (finite) sum: }- „e y w(x) - p(x). 
We shall write Obs(X) = R* and Fact(X) = (Rso)* for the sets of observations and 
factors on X. 


3 Preliminaries on metric spaces 
A metric space will be written as a pair (X, dx), where X isa set and dx : X x X > 


R>o is a distance function, also called metric. This metric satisfies: 


- dx(a,2') =0iffa=27'; 
— symmetry: dx (x, x’) = dx(a’, x); 
— triangular inequality: dx (x, x”) < dx (x, 2’) +dx(a',x"). 


Often, we drop the subscript X in dx if it is clear from the context. We use the standard 
distance d(x, y) = |x — y| on real and natural numbers. 


Definition 1. Let (X, dx), (Y, dy) be two metric spaces. 
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1. A function f: X — Y is called short (or also non-expansive) if: 


dy (f(x), f(2’)) < dx(a,2'), forall xz,x' € X. 


Such a map is called an isometry or an isometric embedding if the above inequality 

< is an actual equality =. This implies that the function f is injective, and thus an 

‘embedding’. 

We write Mets for the category of metric spaces with short maps between them. 
2. A function f: X — Y is Lipschitz or M-Lipschitz, if there is a number M € Ryo 

such that: 


dy (f(x), f(2’)) < M-dx(a,2’), for all x, x' € X. 


The number M is sometimes called the Lipschitz constant. Thus, a short function 
is Lipschitz, with constant 1. We write Met, for the category of metric spaces with 
Lipschitz maps between them (with arbitrary Lipschitz constants). 


Lemma 1. For two metric spaces (X1, dı) and (X2, dz) we equip the cartesian product 
Xı x Xə of sets with the sum of the two metrics: 


d( (21,22), (21, 2)) = dx, (21,24) + dx, (02,24). D 


With the usual projections and tuples this forms a product in the category Meti. O 


The product x also exists in the category Mets of metric spaces with short maps. 
There, it forms a monoidal product (a tensor ®) since there are no diagonals. In the 
setting of [0, 1]-bounded metrics (with short maps) one uses the maximum instead of 
the sum in order to form products (possibly infinite). In the category Met, the 
products X; x Xə with maximum and with sum of distances are isomorphic, via the 
identity maps. This works since for r,s € Rso one as max(r,s) < r + s andr +s < 
2- max(r, s). 


4 The Wasserstein distance between distributions 


This section introduces the Wasserstein distance between probability distributions and 
recalls some basic results. There are several equivalent formulations for this distance. 
We express it in terms of validity and couplings, see also e.g. [1131614]. 


Definition 2. Let (X, dx) be a metric space. The Wasserstein metric d: D(X)xD(X) > 
R>o is defined by any of the three equivalent formulas: 


d(w, w’) = \ T H dx 
tT €depl—1(w,w’) 


_ — re 
= VV weptw Hp (8) 
p, p’ €Obs(X), ppp’ < dx 


= VV Ju EE q-—w' Eql. 


q€ Facts (X) 
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This turns D(X) into a metric space. The operation ® in the second formulation is 
defined as (p ® p’)(x, x’) = p(x) + p'(x'). The set Facts(X) C Fact(X) in the third 
formulation is the subset of short factors X —> R>o. To be precise, we should write 
Facts(X,dx) since the distance dx on X is a parameter, but we leave it implicit for 
convenience. The meet N and joins V in (8) are actually reached, by what are called 
the optimal coupling and the optimal observations / factor. 


In this definition it is assumed that X is a metric space. This includes the case where 
X is simply a set, with the discrete metric (where different elements have distance 1). 
The above Wasserstein distance can then be formulated as what is often called the total 
variation distance. For distributions w, w’ € D(X) it is: 


This discrete case is quite common, see e.g. and the references given there. 

The equivalence of the first and second formulation in is an instance of strong 
duality in linear programming, which can be obtained via Farkas’ Lemma, see e.g. [13]. 
The second formulation is commonly associated with Monge. The single factor q in 
the third formulation can be obtained from the two observations p,p’ in the second 
formulation, and vice-versa. What we call the Wasserstein distance is also called the 
Monge-Kantorovich distance. 

We do not prove the equivalence of the three formulations for the Wasserstein dis- 
tance d(w, w’) between two distributions w, w’ in (8), one with a meet /\ and two with 
a join V. This is standard and can be found in the literature, see e.g. [15]. These three 
formulations do not immediately suggest how to calculate distances. What helps is that 
the minimum and maxima are actually reached and can be computed. This is done via 
linear programming, originally introduced by Kantorovich, see [13[15/3]. In the sequel, 
we shall see several examples of distances between distributions. They are obtained 
via our own Python implementation of the linear optimisation, which also produces the 
optimal coupling, observations or factor. This implementation is used only for illustra- 
tions. 


Example 1. Consider the set X containing the first eight natural numbers, so X = 
{0,1,...,7} C N, with the usual distance, written as dx, between natural numbers: 
dx (n,m) = |n — m|. We look at the following two distributions on X. 


We claim that the Wasserstein distance d(w, w’) is . This will be illustrated for each 
of the three formulations in Definition 2] 


— The optimal coupling r € D(X x X) of w,w’ is: 


T = 5|0,2) +80,3) + 510,6) + §[0,7) + 3] 4,7). 
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It is not hard to see that 7’s first marginal is w, and its second marginal is w’. We 

compute the distances as: 
dew) = T = dx 

1 1 1 

= gax(0,2)+ 5 -dx (0, aS 
—~24346,7,3 _ 18,3 _ 6 _ 15 
=a grat era 3 oe S nag =a 


— There are the following two optimal observations p, p’: X — R, described as sums 
of weighted point predicates: 


p= 1-1, 2-12 3-13 4-14 5-1; 6- 16 7-17 
p =1-1,4+2-124+3-13+4:-144+5-154+6-16+7-17. 


It is not hard to see that (p @ p')(i, j) = p(t) + p'(j) < dx(i, j) holds for all 
i, j E X. Using the second formulation in (8) we get: 


(v Ep) +@' FP’) 


= 3 PO) +5-P(4)+3-P'(2)+3 p3 
r E e E 3 _ 46 _ 30 _ 15 
Sy op eg s 2 8 8 4° 


— 
+ 
Cole 
x 
~ 
D 
wm 
+ 
olor 
SB 
— 
“aI 
7 


— Finally, there is a (single) short factor q: X — R>o given by: 


q=T7-lo t 6-1, t 5-12 } 4-13 t 3-14 } 2-15 } 1-16. 


Then: 


= $-a(0) + 5 -a(4) — ($+ a(2) + $+ a(8) + 4 -a(6) + 3 -a(7)) 
a ae oe (g+44+2) =% 10 _ 20_ 5 _ 15 
2° 2 8 ' 8 ' 8 2 8 4 4 4° 


From the fact that the coupling 7, the two observations p, p’, and the single factor q 
produce the same distance one can deduce that they are optimal, using the formula (8). 


We proceed with several standard properties of the Wasserstein distance on distri- 
butions. 


Lemma 2. In the context of Definition|2| the following properties hold. 


1. For an M-Lipschitz function f: X —> Y, the pushforward map D(f): D(X) > 
D(Y) is also M-Lipschitz; as a result, D lifts to a functor D: Met, —> Met,, 
and also to D: Mets — Mets. 

2. Iff: X >Y is an isometry, then so is D(f): D(X) > D(Y). 

3. Foran M-Lipschitz factor q: X —> R>o, the validity-of-q factor (—) = q: D(X) > 
R>o is also M-Lipschitz. 

4. - each element x € X and distribution w € D(X) one has: d(1|x),w) = w & 

dx(x,—); especially, d(1|x),1)2')) = dx(x, 2"), making the map unit: X > 

D(X) an isometry. 
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. The monad multiplication flat: D?(X) —> D(X) is short, so that D lifts from a 


monad on Sets to a monad on Mets and on Metu. 


. If a channel c: X — D(Y) is M-Lipschitz, then so is its Kleisli extension c >= 


(—) := flat o D(c): D(X) > D(Y). 


. If channel c: X => Y is M-Lipschitz and channel d: Y => Z is K-Lipschitz, then 


their (channel) composite do c: X > Z is (M - K)-Lipschitz. 


. For distributions w;,w; € D(X) and numbers r; € [0, 1] with X$; ri = 1 one has: 


d( Zur: ‘Wi, Xfi” wi) < prie d(wi, wi). 


9. The permutation channel prm: X* — D(X*) from (6) is short. 


Proof. We skip the first two points since they are standard. 


3. 


Let q: X > R>o be M-Lipschitz, then 4 -q: X > Rso is short. The function 
(—) Hq: D(X) > Rso is then also M-Lipschitz, since for w,w’ € D(X), 


Jo q-w' eal =M wk yaw Raya 
<M V |wkp-w'ke| 

p€ Facts (X) 

= M-d(w,w’). 


. The only coupling of 1|x),w € D(X) is 1|£) @w € D(X x X). Hence: 


d(1|z),w) = 1jz)@w H dx = 5 w(x) - dx(z,x') = w = dx (z, —). 
v'EX 


. We first note that for a distribution of distributions 2 € D?(X) and a short factor 


p: X — R>o the validity in Q of the short validity factor (—) = p: D(X) > Rso 
from item[] satisfies: 


2 ((-) Fp) 


X Qw): (w Ep) 
weED(X) 
= 5 5 Q(w) - w(x) - p(x) 
weED(X) rex 
> flat(Q)(a) - p(x) 


rex 
= flat(Q2) = p. 


Thus for 2,2! € D?(X), 
dx (flar(2), flat(.2")) 
= V | flat(Q) Ep- flat(2’) H p| 


p€ Facts (X) 

= |2E ((-) Ep) — H ((-) Ep) | asjust shown 
p€ Facts (X) 

< V |2EQ-NMEQ| by item] 
Q€ Facts (D(X)) 


= dpo (2,2). 
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6. Directly by points (I) and (5). 

7. The channel composite doc = flat o D(d) o c consists of a functional composite of 
M-Lipschitz, K-Lipschitz, and 1-Lipschitz maps, and is thus (M - K -1)-Lipschitz. 
This uses items[I]and (5). 

8. If we have couplings 7; for w;, wi, then }>, r; - 7; is a coupling of )7, r; - w; and 
X; ri - wi. Moreover: 


d( Erri- wi Leiri wt) < (Eirin) dx = ara = dx). 


Since this holds for all 7;, we get: d( ri Wis Atr wt) <i: d(wi, wh). 

9. We unfold the definition of the prm map from (6) and use the previous item in the 
first step below. We also use that the distance between two sequences is invariant 
under permutation (of both). 


dp(xx)(prm(ax), prm(y)) < 5 E 1|£(4))) 


< “7 
K! 
AKSK 
= B pi dx“ Ele) t(y)) by item] 
t: K3K K 
= ` gi dxs (#9) = dxx(a,y). g 
t: K3K i 


Later on we need the following facts about tensors of distributions. 


Proposition 1. Let X,Y be metric spaces, and K be a positive natural number. 


1. The tensor map ®: D(X) x D(Y) > D(X x Y) is an isometry. 
2. The K-fold tensor map iid|K]: D(X) + D(X *), given by iid[K](w) = w* = 
w8: Qu, is K-Lipschitz. Actually, there is an equality: d(w* , p“) = K-d(w, p). 


Proof. 1. Let distributions w,w’ € D(X) and p,p’ € D(Y) be given. For the in- 
equality dp(x)x Dy) ((w, p), (w, p')) < dpxxy) (w @p,w' Q p') one uses that a 
coupling T € D((X x Y) x (X x Y)) ofw@p,w' @p' € D(X x Y) can be turned 
into two couplings 71,72 of w,w’ and of p, p', namely as 7; = D(m; x mi) (T). 
For the reverse inequality one turns two couplings 71, 72 of w,w’ and p, p’ into a 
coupling 7 of w @ p,w’ Q p' via T := D((m X T1, T2 X T2)) (Ti @ T2). 

2. Forw, p € D(X) and K € N, using the previous item, we get: 


dpixx) (w*, p*) dpr ((w,-.-5); (P: -0)) K -dpx)(w, p). LJ 


5 The Wasserstein distance between multisets 


There is also a Wasserstein distance between multisets of the same size. This section 
recalls the definition and the main results. 


Drawing from an Urn is Isometric 111 


Definition 3. Let (X,dx) be a metric space and K € N a natural number. We can 
turn the metric dx: X x X —> Rso into the Wasserstein metric d: M[K](X) x 
M{[K](X) > Rso on multisets (of the same size), via: 


d(y, p") = \ Firn(r) = dx 
TEdcepl™! (p,p') 
\ dy x (a, x’) (9) 


w€acc—!(y), x Eace—1(p’) 
t 
y dx (£i, £4). 


xEacc™! (p), x’ Eace—!(y’) OSi< K 


[S] 
Ale N= 


All meets in (9) are finite and can be computed via enumeration. Alternatively, one 
can use linear optimisation. We give an illustration below. The equality of the first two 
formulations is standard, like in Definition [2] and is used here without proof. There is 
an alternative formulation of the above distance between multisets that uses bistochastic 
matrices, see e.g. (216), but we do not need it here. 


Example 2. Consider the following two multisets of size 4 on the set X = {1,2,3} C 
N, with standard distance between natural numbers. 


yp = 3/1) +12) ye = 2|1) + 1/2) +113). 
The optimal coupling r E€ M[4](X x X) is: 
T = 2|1,1) +1]1,2) +1[2,3). 
The resulting Wasserstein distance d(y, y’) is: 


Firn(r) = dx = 4- dx(1,1) + 4 - dx(1,2) +4 -dx(2,3) = ¢-1+4-1= 4. 


Alternatively, we may proceed as follows. There are (p) = wa = 4 lists that accu- 
mulate to y, and (y’) = a = 12 lists that accumulate to y’. We can align them all 


and compute the minimal distance. It is achieved for instance at: 
4+ dyxa((1,1,1,2),(1,1,2,3)) f 4. (0+0+1+1)=2=4. 


Lemma 3. We consider the situation in Definition|3] 


1. Frequentist learning Flr: M|{K](X) > D(X) is an isometry, for K > 0. 

2. For numbers K,n > 1 the scalar multiplication function n - (—): M[K]|(X) > 
M[n - K](X) is an isometry. 

3. The sum of distributions +: M[K|(X) x M[L](X) — MIK + L](X) is short. 

4. If f: X — Y is M-Lipschitz, then M[K](f): M[K](X) > M[K](Y) is M- 
Lipschitz too. Thus, the fixed size multiset functor M|K] lifts to categories of metric 
spaces Mets and Metz. 

5. For K > 0 the accumulation map acc: XK — M[K](X) is +-Lipschitz, and 
thus short. 
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6. The arrangement channel arr: M[K](X) => X* is K-Lipschitz; in fact there is 
an equality d(arr(y), arr(y’)) = K - d(y, y’). 


Proof. 1. Vianaturality of frequentist learning: if r € M[K](X x X) is a coupling of 
yy’ E€ M[K](X), then Flrn(r) € D(X x X) is a coupling of Firn(y), Firn(y’) € 
D(X). This gives d(y, y’) < d(Firn(vy), Flrn(y’)). The reverse inequality is a bit 
more subtle. Let co € D(X x X) be an optimal coupling of Firn(y), Firn(y’). 
Then, since any coupling r E€ M[K](X x X) of y, y’ gives, as we have just seen, 
a coupling Flm(r) € D(X x X) of Firn (p), Flrn(y’), we obtain, by optimality: 


d(Firn(y), Fimn(y')) = o = dx < Firn(r) H dx. 


Since this holds for any coupling 7, we get d(Flrn(y), Firn(y’)) < d(y,¢’). 
2. For multisets y, yp’ E€ M{[K](X), by the previous item: 


duty x) (9, ¥") = docx) (Firn(¢), Firn(y’)) 
= dpcx)(Fimn(n- p), Firn(n- y’)) 
= dmm- K(x) (n p,n- p’). 


3. For multisets y, yp’ € M[K](X) and w, Y’ € M[L](X), using Lemmal2|(8}. 


d pty +y) 
oe +~), Flrn(y’ + W) 

im (yp) + K ` Firn (4), Äg” Firn((p') + aes 
Xe ae ale ')) + ey - d(Flrn (4), Fira (4’)) 
K+L -d(y,¢' ) + KT : d(w,¥") 
dpe’) +d(v,¥) 
a(t, b) (wp). 


4. Let f: X — Y be M-Lipschitz. We use that frequentist learning Firn is an isome- 
try and a natural transformation M[K] = D. For multisets y, yp’ € M[K](X), 


dman (M( fy(e ), M(f)(~ ‘)) 
E dp) (Ein ( MC) (¢)), Flim(M(1)(¢'))) 


IEB- 


l 
a 


- Firn("), ) 


IEMA 
Site 


a IA 


= dny) | D(f)( Firn(g)),D(f)(Firn(y")) ) by naturality of Firn 
< M - dp,x)(Firn(¢), Firn(¢’)) by Lemma|2|(I) 
du 

= dm (p y’). 


5. The map acc: X* — M[K](X) is +-Lipschitz since for y, y’ € X“, 


d(ace(y), ace(y’)) = dyx (a, x’) 


w€acc—!(acc(y)), x’ Eacc—!(acc(y’)) 


-dxx(y,y’). 


IA 
Ale OR] 
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6. For fixed y, y’ € M[K](X), take arbitrary x € acc~'(p) and x € acc~1(y’). 
Then: 


dp(xk) (arr(acc(a)), arr(acc(z’))) 
dp(xk) (prm (cx), prm(z’)) 
dx x (x, x’) by Lemma|2](9). 


Since this holds for all x € acc~!(y), 2’ € acc~'(y’) we get an inequaltiy 
dp(xx) (arr(y), arr(y’)) < K-daxyxy(¢, g’), see Definition This inequality 
is an actual equality since acc, and thus D(acc), is +-Lipschitz: 


dD(xK) (arr(9), arr(%')) 


IE |l 


IA 


dmx lpp) = domix (1 ¢), - )) 
= domix) (D(ace) (arr(y)) ,D(ace) (arr(y)) ) 
< %°dy(xx)(arr(y), arr(y’) E 


6 Multinomial drawing is isometric 


Multinomial draws are of the draw-and-replace kind. This means that a drawn ball is 
returned to the urn, so that the urn remains unchanged. Thus we may use a distribu- 
tion w € D(X) as urn. For a draw size number K € N, the multinomial distribution 
mn{K|(w) € D(M[K](X)) on multisets / draws of size K can be defined via accu- 
mulated sequences of draws: 


mn[K](w) = D(acc)(w*) 
= D(acc) (iid[K](w)) 


= X (w [Le] 


pe M[K](X) eX 


(10) 


We recall that (y) = Teas is the number of sequences that accumulate to a multiset 


z o) 
/ draw p € M[K]|(X). A basic result from [8] Prop. 3] is that applying frequentist 


learning to the draws yields the original urn: 
Firn >= mn[K](w) = w. (11) 
We can now formulate and prove our first isometry result. 


Theorem 1. Let X be an arbitrary metric space (of colours), and K > 0 be a positive 
natural (draw size) number. The multinomial channel 


D(X) mn[K] 


D(M[K|(X)) 

is an isometry. This involves the Wasserstein metric (8) for distributions over X on the 
domain D(X), and the Wasserstein metric for distributions over multisets of size K, 
with their Wasserstein metric (9), on the codomain D(M[K](X)). 
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Proof. Let distributions w,w’ € D(X) be given. The map mn |<] is short since: 
dpcaatx\(x) (mn[K](w), mn[K](")) 

TO és 5 

T dpm) (Dlace)(iid[K|(w)), D(acc) itd KI) 
dogr (iia [K](w), iid [K] (w) by Lemma p] (5) 
-K - dpx (w, w’) by Proposition [I] (2) 


= dyxy(v, w’). 


| 
al Ale 


There is also an inequality in the other direction, via: 
dox (w, w) E doo) (Firn >= mn[K] (w), Flm >= mn[K](w’)) 
< dvimixx) (ma[K](w), m[K](w')). 


The latter inequality follows from the fact that frequentist learning Firn is short, see 
Lemma[3|(I), and that Kleisli extension Firn >= (—) is thus short too, see Lemmal2|(6). 
Lj 


Example 3. Consider the following two distributions w, w’ € D(N). 


w = 4/0)+2]2) and w= 4/1) +4]2) with d(w,w") = 4. 


This distance d(w, w’) involves the standard distance on N, using the optimal coupling 
510,1) + 312,1) + 412,2) € DIN x N). 
We take draws of size K = 3. There are 10 multisets of size 3 over {0, 1,2}: 
pı =3|0)  2=2|0)+1|1) = v3 =1]0)+2/1) %4 = 3/1) 
ps =2|0) +12) ye =1|0) +1|1) +1|2) %7 = 2|1) +1|2) 
ps =1|0)+2|2) yə =1|1)+2|2) %10 = 3|2). 
These multisets occur in the following multinomial distributions of draws of size 3. 


mn[8](w) = a791) f 3125) ] Slps) + 7| v10) 


mn[3](w’) = ilya) + Elor) + 2] G0) + iloi) 


The optimal coupling r € D(M[3](N) x M[3](N)) between these two multinomial 
distributions is: 


T= dy onpa) +a ws 94) +i pio p10) + 3% | es, er) 


5 
+ 75 


Ys, p7) +2 les, o + 35 | P10, yr . 
We compute the distance between the multinomial distributions, using dm = d mM{3](N)- 
d(mn([3](w), mn[3](w")) = TH dm 
= $- dm(p1, p4) + Hy -dm (Hs, 44) + $- dm(p10, p10) + Fe -dm (Hs, 97) 
+7 dm (ps, 97) +3 dm (p8, 9) + 316 ` dm (V10, 27) 


-14.14 32.141.042.24} 5.243.144} 37.21 
— 27 1+ z6 1+ 3 O+ zig 3 + 7 3 t3 3 + 216 3 ~ o p 
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As predicted in Theorem |1| this distance coincides with the distance d(w,w’) = 4 
between the original urn distributions. One sees that the computation of the distance 
between the draw distributions is more complex, involving “Wasserstein over Wasser- 
stein’. 


7 Hypergeometric drawing is isometric 


We start with some preparatory observations on probabilistic projection and drawing. 


Lemma 4. For a metric space X and a number K, consider the probabilistic projec- 
tion-delete PD and probabilistic draw-delete DD channels. 


et SS De) MIK + 1](X) == D(M[K](X)) 


They are defined via deletion of elements from sequences and from multisets: 


1 


PD(#1,..-,2K41) = —_|@1,..-, B41, itl- 2K 41) 
iick Etl 
p(x) 
DD(w) := 5 gp Vh 
x€supp(w) 
= So Arn(w)(x)|p-1]2)). 
xe supp(w) 


Then: 


I. <acc> o PD = DD © <acc>; 

2. Flrn >= DD(w) = Firn(w); 

3. PD is Raq -Lipschitz, and thus short; 
4. DD is an isometry. 


Proof. The first point is easy and the second one is [8} Lem. 5 (ii)]. 
3. Fora,y € X*+1, via Lemmal2|(8) and (4, 


1 
d(PD(«), PD(y)) =d 5 K ISe Rie Ti SKH) 
1<i<K41 
5 E E E TE 
K+1 
1<i<K+1 
1 
S > K+1 (Atay. tinti EEH N 
1<i<K41 
1 Vase Birds Hitt YKY) 
1 
= Kai ix“ (Cee eee ee Tart 
1<i<K41 
E Minas Betas ++ T E) 
1 
= l pyg E ax(eisu) 
1<i<K41 
K 
——— -dyry (x,y). 
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4. Via item[I]we get: 


<acc> o PD o arr = DD © <acc> o arr = DD © unit = DD. (x) 


Now we can show that DD is short: for Y, Y’ € M[K + 1](X) 


dp(m{K(x)) (PDY), DD") 
2 dp(M[K](X)) (D(ace) (PD »= arr(7)), D(acc) (PD »= arr(w")) 


2 a dox on (Pp y= arr(Y), PD >= arr(w")) 
< KOR a — (W), arr(")) 

= KH (K +1). AK +1](X yy, ro) 

= dmx) (Y, Y). 


For the reverse inequality we use item [2]and the fact that FIrn is a short: 


dD(MIK](X)) (DD(y), DD(w')) 

> dpemixi(x)) (Fim >= DD(%), Fim >= DD(w")) 

= dpx) (Flen(i)), Flra(4')) 

= dmx) (Y, Y’). m 


The hypergeometric channel hg|K]: M[L](X) > D(M[K](X)), for urn size 
L > K, where K is the draw size, is an iteration of draw-delete’s, see Thm. 6]: 


ita 


hg|K](v) = DD e -- -o DD = 5 lp), (12) 
L—K times pEM[K](X), p<u (x) 


where (X) = Maex (a). 


Theorem 2. The hypergeometric channel hg[K|: M[L](X) > D(M[K](X)) de- 
fined in 23, for L > K, is an isometry. 


Proof. We see in that hg[K] is a (channel) iteration of isometries DD, and thus 
of short maps; hence it it short itself. Via iterated use of Lemma [4] (2) we get Flrn »= 
hg|K](q) = Firn(2). This gives the inequality in the other direction, like in the proof 


of Lemma/4] (2): 
dm (Y, Y) = docx) (Flen (Y), Firn(v’)) 
= domirx p (Pm > i J(Y), Firn > hg[KI(W)) 
< dpwmatxy(xyy (hg [A] (Y), he[K](Y")). m 


The very beginning of this paper contains an illustration of this result, for urns over 
the set of colours C = {R, G, B}, considered as a discrete metric space. 
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8 Pólya drawing is isometric 


Hypergeometric distributions use the draw-delete mode: a drawn ball is removed from 
the urn. The less well-known Pólya draws [7] use the draw-add mode. This means that 
a drawn ball is returned to the urn, together with another ball of the same colour (as the 
drawn ball). Thus, with hypergeometric draws the urn decreases in size, so that only 
finitely many draws are possible, whereas with Pólya draws the urn grows in size, and 
the drawing may be repeated arbitrarily many times. As a result, for Polya distributions 
we do not need to impose restrictions on the size K of draws. We do have to restrict 
draws from um v to multisets p € M[K](X) with supp(y~) C supp(v) since we can 
only draw balls of colours that are in the urn. Pólya distributions are formulated in terms 
(ntm—1) = Qim- forn > 0. This multi- 


m m!-(n—1)!? 
choose number E) is the number of multisets of size m over a set with n elements, 


see [9[10] for details. 


of multi-choose binomials ((”)) := 


pl[K](v) = 5 e< |p), (13) 
pEM[K](X), supp(y)Csupp(v) (x) 


wire (D= TT oe 


Theorem 3. Each Pólya channel pl|K]: M{L|(X) > D(M[K](X)), for urn and 
draw sizes L > 0, K > 0, is an isometry. 


Proof. One inequality follows by exploiting the equation Flrn y= pI|K] (Y) = Firn(w) 
like in previous sections. The reverse inequality, for shortness, involves a draw-store- 
add channel of the form: 


M{L](X) x M[N](X) —— 4 D(M[L](X) x MIN + 1](X)) 
defined as: 


DSA(v,~) = > Flm(v + ¢)(2) |v, + 12) 
x€supp(v+y¢) 


= 1|v)®@ 5 Firn(v + y)(x)| p + 1]x)) 
x€supp(v+y) 
With some effort one shows that this channel DSA is short and that the Polya channel 
can be expressed via iterated draw-store-add’s, namely as: 
pl[K](v) = D(m2)((DSA o- -o DSA )(v,0)), 
a 
K times 


where 0 € M[0](X) is the empty multiset. This makes the Pólya channel p1 [K] short, 
and thus an isometry. LJ 
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We illustrate that the Pólya channel is an isometry. 
Example 4. We take as space of colours X = {0, 10,50} C N with two urns: 
vı = 3/0) + 1/10) v2 = 1/0) + 2]10) + 1]50). 


The distance between these urns is 15, via the optimal coupling 1|0,0) + 2|0,10) + 
110, 50), yielding + - (0 — 0) + 4 - (10 — 0) + 4 - (50 — 10) = 5 + 10 = 15. 
We look at Pólya draws of size K = 2. This gives distributions: 


pl[2}(v1) = 3|2I0)) + = |110) +1/10)) +4 |2|10)) 
pl[2|(v2) = 45] 210)) + 4 |110) + 1/10)) + $ |2/10)) + $ |110) + 1/50) 
+ 4|1|10) + 1|50)) + $ |2150)) 


We compute the distance between these two distributions via the last formulation in (8), 
using the optimal short factor p: M[2](X) — R>o given by: 


p(2|0)) = 0 p(110)+1]10)) = 5 p(2|10)) = 10 
p(1|0) +1]50)) = 25 p(1|10) + 1|50)) = 30 p(2|50)) = 50. 
Then: 
pil (v) Ep = 2-04+3-54+ 4-10 = 3 
pl[2|(v2) Ep = 0442-543 -104+ 4% -254+-2-30+5-50 = &. 


w 


As predicted by Theorem[3| the distance between the Pólya distributions then coincides 
with the distance between the urns: 


a(p1[2](v1), pll2\(v2)) 


II 
5 

— 
N 

e 
S 
x 


(v1) E p= pl[2](v2) E p| 
3 _ i =15= d(v1, və). 


9 Conclusions 


Category theory provides a fresh look at the area of probability theory, see e.g. 
or for an overview. Its perspective allows one to formulate and prove new results. 
This paper demonstrates that draw operations, viewed as (Kleisli) maps, are incredi- 
bly well-behaved: they preserve Wasserstein distances. Such distances on urns filled 
with coloured balls are relatively simple, starting from a ‘ground’ metric on the set of 
colours. But on draw distributions, the distances involve Wasserstein-over- Wasserstein. 
This paper concentrates on drawing from an urn. A natural question is whether other 
probabilistic operations, as Kleisli maps, preserve distance. This is a topic for further 
investigation. 
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Abstract. In this paper, we extend diagrammatic reasoning in monoidal 
categories with algebraic operations and equations. We achieve this by 
considering monoidal categories that are enriched in the category of 
Eilenberg-Moore algebras for a monad. Under the condition that this 
monad is monoidal and there is an adjunction between the free algebra 
functor and the underlying category functor, we construct an adjunction 
between symmetric monoidal categories and symmetric monoidal cate- 
gories enriched over algebras for the monad. This allows us to devise 
an extension, and its semantics, of the ZX-calculus with probabilistic 
choices by freely enriching over convex algebras, which are the algebras 
of the finite distribution monad. We show how this construction can be 
used for diagrammatic reasoning of noise in quantum systems. 


1 Introduction 


Monoidal categories are one way of generalizing algebraic reasoning and they can 
be used to draw intuitive diagrams that encapsulate this reasoning graphically. 
That monoidal categories are a powerful abstraction has been demonstrated 
in countless areas, such as linear logic [19] or quantum mechanics [I], just to 
name a few, and are amenable to graphical reasoning [47] with diagrammatic 
languages such as the ZX-calculus [I5]. Another abstraction of algebraic reason- 
ing are monads [3]37/43] and their algebras, or representations thereof [21)86], 
which are distinct from monoidal categories in that identities (like associativ- 
ity) always hold strictly and they allow rather arbitrary algebraic operations. In 
this paper, we set out to combine these two approaches into one framework, in 
which monoidal category diagrams can be composed not only sequentially and 
in parallel with a tensor product but also with additional algebraic operations. 

One such operation is the formation of convex combinations, which can be 
used to create a probabilistic mix of two or more diagrams. This occurs, for 
instance, when reasoning about the behaviour of noise in quantum circuits. 
Figure |1| shows on the left two quantum logic gates, one called G and one 
called Æ that, respectively, model the wanted behaviour and a possible error. 
These two gates are mixed, where G gets a probability of 0.9 and F of 0.1. 
The trapezoids delimit the combination of the gates, and A and B are the in- 
put and output types of the gateq!| In monoidal categories, the gates in the 


1 We read diagrams from top to bottom. 
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Fig. 1: Left: Probabilistic mix of a gate G with an error E. Right: Interaction 
of tensor and convex sum, where double wires visually indicate a tensor product 


picture represent morphisms G, E&E: A — B and our aim is to interpret the 
trapezoid block as a convex sum G +ọ.9 E of these morphisms, where we de- 
fine G+, E = pG + (1 — p)E. Such sums should also nicely interact with the 
tensor product. For instance, if Gi: A > B and G2: C > D are gates, then an 
identity such as (Gy +0.9 E) & Go = (Gy ® G2) +0.9 (E ® G2) should hold for 
these morphisms of type A ® C > B & D, see Figure[]] on the right. Having an 
operation to form convex combinations together with intuitive identities enables 
reasoning about, for example, probabilistic combination and noise in quantum 
circuits. 

The difficulty lies in combining monoidal diagrams with algebraic operations 
such that the algebraic identities and the monoidal identities interact coherently. 
We will handle this difficulty by using enriched monoidal categories, where the 
enrichment yields the algebraic operations and the monoidal structure the paral- 
lel composition. More precisely, we will assume that the algebraic theory is given 
by a monad T and that the monoidal categories are enriched over the Eilenberg- 
Moore category Alg” of algebras for this monad. Our aim in this paper is to 
construct for an arbitrary monoidal category C an Alg” -enriched monoidal cat- 
egory FC that is free in the sense that there is an inclusion t¢: C > (FC)o into 
the underlying category of FC and for every Alg’-enriched monoidal category 
D and monoidal functor G: C — Dp, there is a unique Alg” -enriched monoidal 
functor Go: FC > D that makes the following diagram commute. 


(FC)o cee Do 
al P 


This free construction does not work for all monads, but we show that the free 
enrichment always exists for monoidal Set-monads whose free T-algebra functor 
is left adjoint to the underlying category functor Alg” (I ,—): Alg?” — Set for 
I the monoidal unit of Alg”. 


Contributions 

Specifically, we contribute a construction for free enrichment over algebras for 
some monoidal monads in Theorem[i]and Corollary[1| We also show how the en- 
richment preserves symmetric monoidal structure in T heorem[3] and Corollary[2} 
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Given this construction, we demonstrate how a graphical language for reasoning 
in monoidal categories can be enriched with the free algebras for these monads, 
which enables diagrammatic reasoning of the interaction between the sequen- 
tial and parallel compositions with the algebraic structure. We show how the 
theory can be applied to obtain convex combinations of ZX-diagrams and what 
the resulting identities of diagrams are. By exploiting the mapping property of 
the free enrichment, we automatically obtain sound interpretations of these op- 
erations and identities. Lastly, we describe how we can use the enrichment of 
ZX-diagrams to reason about noise in quantum systems. 


Related Work 

ZX-diagrams are universal in the sense that they can in principle represent any 
linear map between Hilbert spaces of dimension C?” [I5]. Indeed, sums and 
linear/convex combinations [25/49] of ZX-diagrams can be encoded within the 
language, but in practice these representations oftentimes lead to either very 
large diagrams or to diagrams that do not reveal upon visual inspection the 
(linear/convex) structure that the diagram is representing. This, in return, di- 
minishes the advantages gained by reasoning in terms of abstract graphical rep- 
resentations. Our perspective of using enrichment keeps the abstraction barrier 
and thus makes reasoning about convex combinations of diagrams tractable. In 
general, our theory also covers the recently developed linear combinations of 
ZX-diagrams such as [50J39] and other, so far unexplored, algebraic operations 
such as those of join-semilattices. Moreover, the identities that have to be crafted 
carefully by hand and proven to be sound fall automatically out of our theory. 
Other related work is that of Sheet diagrams [16] and Tape diagrams [5], recently 
developed graphical languages for rig categories, which are categories with two 
monoidal structures — one for addition and one for multiplication. 


Outline 

The paper is organised as follows. We start by introducing notation and recalling 
some background of enriched and monoidal categories in Section |2| In Section 
we establish the necessary theory to define categories enriched over Eilenberg- 
Moore algebras and we construct a free enrichment over those algebras. Our next 
step in Section Blis to extend these definitions and the free construction to also 
include monoidal structures on categories, which ensures that these enrichment 
and monoidal structure coherently interact. Section Blis devoted to applying our 
theory to enrich ZX-diagrams with convex sums to reason about probabilistic 
processes such as quantum noise. We conclude the paper with directions for 
future work in Section [6] 


2 Background 


In this section, we recall some terminology from category theory [8]34]35/45] and 
introduce some notation. We denote the collection of objects of a category C as 
|C], and the morphisms from object A to B as C(A, B). A monoidal category 
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(C,&, T) is a category C together with a functor & : C x C > C called the 
tensor product and an object I € |C| called the tensor unit subject to some 
conditions [28]. We will often refer to a monoidal category (C,®, J) as just C. 
A monoidal category is a symmetric monoidal category (SMC) when it also has 
a braiding o4,p : A® B > B & A such that og,4°0,4,8 = Idyge that is also 
subject to coherence conditions |28|. 

Given a monoidal category (V, x,*), a V-(enriched) category C consists of 


— a class |C| of objects, 

— for each pair A,B € |C], an object C(A, B) € |V| that we refer to as the 
hom-object, 

— for objects A, B,C € |C|, a composition morphism o: C(B,C) x C(A, B) > 
C(A, C) in V, and 

— for all A € |C], an identity element ja: * + C(A, A) 


subject to associativity and unit axioms [28]. We say that V is the base of en- 
richment for C. A way to look at the above definition is that we construct a 
V-enriched category C by identifying morphisms of some category C as ob- 
jects from V, which we are able to compose by using the tensor product of V. 
The most well-known example is that of locally small categories, in which the 
morphisms between two objects form a set, and thus we can see them as ob- 
jects in the monoidal category (Set, x, *) for x the Cartesian product and * the 
singleton set. 

With a suitable definition of V-functors and V-natural transformations, V- 
categories organise themselves into a 2-category [28], denoted by V-Cat. For an 
SMC (V, x, *), V-Cat is also an SMC as follows. We define for V-categories C 
and D a V-category C & D with objects |C ® D| being the categorical product 
and hom-objects (C & D)((A, B),(C,D)) = C(A,C) x D(B, D). The unit is 
given by j(a,B) = * = * X * waX*®, C(A, A) x D(B, B) in terms of the units 
u of C and v of D. Similarly, one also defines the composition for C & D in 
terms of the composition morphisms of C and D, appealing to the symmetry 
in V [28] Sec. 1.4]. The tensor product also extends to V-functors and V-natural 
transformations, which makes it a 2-functor. Finally, one defines J to be the unit 
V-category with one object 0 and J(0,0) = * and we thus obtain, with suitable 
definitions of associators etc., a symmetric monoidal 2-category (V-Cat, &, I). 

Most of the categories we are interested in are also dagger-compact categories 
({-CC). These are SMCs (C, &, I) with some additional structure. First, they 
are equipped with an endofunctor + : C — C, that satisfies (Ida)? = Ida, 
(go f)i = ft o gt, (FÌ) = f, and (f @g)' = ft @ gt. And secondly, for every 
object A there exists a dual A* such that there exists unit n4 : I > A® A* and 
counit c4 : A* Q A— I morphisms subject to some conditions [20]. 

We are interested in categories that let us reason about quantum mechanics. 
One of them is FdHilb, the category of finite dimensional Hilbert spaces of 
the form C” and linear maps as morphisms. The category Qubit is the (full) 
subcategory of FdHilb with objects Hilbert spaces of the form C?” and linear 
maps. Similarly, the category CPM(Qubit) [48] has objects C?” and morphisms 
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completely positive linear maps between them [13]. We usually work in Qubit 
when reasoning about pure quantum evolutions and in CPM(Qubit) when 
impure quantum evolutions (such as noise) can take place. All of these categories 
are {-CC, with the monoidal structure ® given by the usual Kronecker product 
of vector spaces and the dagger f being the conjugate transpose. 


3 Algebraic Enrichment 


In this section, we are going to recall the concept of monoidal and affine monads, 
and discuss some properties of the Kilenberg-Moore category of a monad. We 
also start applying the Distribution monad and the Multiset monad to running 
examples that will be of interest in later sections. 

The Distribution monad (D, p, nf] contains the functor D : Set — Set that 
maps a set A to the set D(A) of (finitely supported) probability distributions 
over elements of A. We write probability distributions as formal convex sums: 
>>, Pala] € D(A) such that a € A,pa € [0,1], and 5°, Pa = 1. D acts on 
a morphism f by simply applying f to the underlying set: (Df)()°, Pala]) = 
>), Palf(a)]. The unit of the monad is the map 7 : A > D(A) : a+ 1{a] 
(the Dirac distribution), and the multiplication jy “flattens” a distribution of 
distributions by multiplying the probabilities together: u : D(D(A)) > D(A): 
dig Pala Gala] => 20, rala] where ra = D1, Paga B3]. 

The functor D is also a monoidal functor, which makes (D, u,n) a monoidal 
monad. In particular, this means that there exists a map: 


V : D(A) x D(B) > D(A x B): (Sra ot) ++ ` papol(a,)] 
a b a,b 


for every A, B € |Set]. 

A monad T : C > C is affine if there is an isomorphism T(*) = x for * the 
terminal object of C [24]. This is the case for D. 

If D is a monad for expressing convex combinations of elements of a set, the 
Multiset monad M is its analogue for linear combinations with coefficients over 
some semiring. 

We recall that given any monad T in C we can construct its Eilenberg-Moore 
category, with objects T-algebras of the form (A, œa) for A € |C] and T-action 
aa : T(A) > A such that a4 o T(aa) = aa o ua and aa ona = Ida. Algebra 
homomorphisms f : (A,a@4) > (B,ag) are morphisms of the underlying objects 
f : A —> B that commute with the action: f oaa = agoT(f). The identity and 
composition follow from the ones for the underlying objects [35]. 

For a monad T on C, we have that Alg is complete whenever C is com- 
plete. Cocompleteness is not as immediate, but if C = Set then Alg? is also 
cocomplete [8122]. This makes Alg” over monads on Set a complete and cocom- 
plete category and in particular, Alg” has reflexive coequalizers, which we use 
to define the tensor product of algebras. 


2 We will often refer to a monad (T, u,n) as T. 
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Q, I), the tensor 


When T is a monoidal monad on a monoidal category (C, 
b), is (if it exists) 


product of T algebras (A,a),(B,b), denoted (A,a) @7 (B, 
defined as the coequalizer diagram [46[10} 


wF(V) 
F(T(A) ®T(B)) °? F(A@ B) —> (A,a) QT (B,D), (1) 
F(ab) 


where F : C > Alg” : A œ> (T(A), p) is the left adjoint to the forgetful 
functor U : Alg’ + C : (A,a4) +> A that maps objects to their free algebras 
over T. Given that we need Alg to be monoidal in order to use it as a base 
of enrichment, diagram (1) above is a convenient representation of the tensor 
product of algebras. The rest of the structure to make Alg? a (symmetric) 
monoidal category follows under certain conditions, in particular a (C, @, I) 
is a closed (S)MC and the coequalizer (1) exists for all algebras (A, a), (B, b) [6]. 

We can define (symmetric) monoidal structure in the category of free algebras 
as follows. Using for the category of free algebras over a monoidal monad, 
we have that the following diagram forms a coequalizer. 


wF(V) 
F(TT(A) & TT(B)) a F(T(A) @T(B)) “2™, F(A@B) (2) 
HBH 


Therefore, (T (A), u) @7 (T(B), u) = F(A & B) [46] Prop. 2.5.2]. The monoidal 
unit is IT = (T(J), p), while the associator, unitor, and symmetry (if present) are 
the images of the ones in (C, &, I) under F. We then have that (Alg’, 87, IT) 
is the (symmetric) monoidal category of free T-algebras. 

A functor F : (Vi, 8v1, Ivi) > (V2, 8v2, Iv, ) between two monoidal cat- 
egories can be lifted to a 2-functor F, : Vı-Cat —> V»-Cat [7]. This is called 
a change of enriching, where we turn a Vj-category into a Vo-category. In- 
deed, given a V,-category C, we can construct the V2-category F.C by defining 
|F’.C| := |C] and, for every A,B € |F.C|, the hom-objects are F,C(A, B) := 
F(C(A, B)) with composition and identity element following from the ones in C 
under F. For a symmetric monoidal category (V,®,J), an important instance 
is the functor V(I,—).: V-Cat — Cat called the underlying category functor 
and it is denoted by (—)o. 

The following lemma states explicitly the case when one of the enriching 
categories is Set. 


Lemma 1 ([7] Prop. 6.4.7]). Let (V,®,I) be a closed symmetric monoidal 
category with coproducts. Then the hom-functor V(I,—): V — Set has a left 
adjoint F that sends a set X to F(X) = ||yI, the X-th fold copower of I. 
Moreover, F is a strong morphism of symmetric monoidal categories and the 
induced 2-functor F, is left-adjoint to the underlying category functor (—)o. 


Theorem 1. A monoidal monad T on (Set, x,*) endows the category of T- 
algebras Alg” with a bicomplete (complete and cocomplete) closed SMC struc- 
ture. This allows to ije the free-forgetful adjunction of Lemma[{1] as a change of 
enriching between Alg’ -categories and Set-categories for a monoidal T. 
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Proof. The proof follows from Lemma |1|and previous arguments. Given that 
Alg” for T a monad on Set is bicomplete, then coequalizer exists and we 
can define tensor products of algebras. We can then make Alg” a symmetric 
monoidal category given that Set is closed symmetric monoidal. Finally, Alg” 
can be made into a closed category following [32] given that Set has equaliz- 
ers. We can then use Lemma |1|to create a change of enriching between Alg’- 
categories and Set-categories. 


Corollary 1. Let T be a monoidal monad on Set defined by a free-forgetful 
adjunction U : Alg” = Set: L. If L is naturally isomorphic to the functor F 
from Lemma that is (T(—), u) = JI; (TI, u), then the induced 2-functor Ly is 
left adjoint to the underlying category functor (—)o. This lets us use Theorem [1] 
to enrich locally small categories with the free algebras over T. The condition 
L S F holds, in particular, when T is an affine monad. 


Proof. Whenever we have that L = F, the enrichment over free T-algebras 
comes simply from substituting F with L in Lemma |1| and Theorem To 
see that this condition holds when T is affine, we construct hom isomorphisms 
Alg’ (L(X),Y) S Set(X,U(Y)) = Set(X, Alg’(I7,Y)) = Alg’ (F(X), Y) for 
some X € |Set|,Y € |Alg’|, with the second and last isomorphism coming 
from their respective adjunctions. The remaining one is due to T(*) S x, which 
allows us to get algebra homomorphisms h : (T(*), u) +> (T(Y), u) from maps 
h’ :* + Y, while the other direction just requires to forget the homomorphism 
structure. 


Let us construct an example for Theorem [I] and relate it to graphical lan- 
guages. If we have a locally small monoidal category C with morphisms f,g : 


A — B,h: B — C, represented graphically as |s], Js], [a], we can freely 


enrich C over Alg? following the change of enriching category method above. 
Then, we can realize graphically a probabilistic process involving f and g with 
probability 0.9 and 0.1 respectively, followed by applying h deterministically 
(that is, it occurs with probability 1) afterwards as follows. 


0.9 Aoi 

filg 

we (3) 
h 


Intuitively, we distinguish between probabilistic and deterministic processes 
by having the former enclosed within distribution brackets (in the same way as 
we would represent them as a formal sum 0.9[f] + 0.1[g]), that we choose to 
depict as trapezoids in this paper. Deterministic processes are depicted without 
the bracket enclosing mostly as syntactic sugar, otherwise they would simply 
have a single choice with probability 1. We can see how wires can have weights 
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inside this environment, and how each wire represents a probabilistic choice. 
Intuition also tells us that we could for example rewrite the diagram above 
by distributing h over the two probabilistic branches. We will discuss in later 
sections which graphical rules capture the interactions present in these enriched 
categories. 

It is natural to then ask if our enriched category C maintained its monoidal 
structure, and if other desired properties (such as braiding and symmetry, if 
present) would still hold too. We will address this in the next section. 


4 Enriched Monoidal Categories 


Recall from Section P] that a symmetric monoidal category V gives rise to a 
symmetric monoidal 2-category (V-Cat,@,J) of V-categories. This structure 
allows us to define an enriched (symmetric) monoidal category to be a (sym- 
metric) pseudo-monoid in V-Cat [18], which amounts to the following explicit 
definition [83]38]. Let us denote by S the symmetry isomorphism of V-Cat. A 
symmetric monoidal V-category is a tuple (C, ©, U, a, A, p, o) consisting of: 


— a V-enriched category C 

— a V-functor U: I => C 

— a V-functor ©: C&C + C 

— V-natural isomorphisms a: © 0(© ® Ide) > © o (Ide & ©) (associator), 
à: ©o(U Ide) > Ide (left unitor), p: ©o(Ide &U) > Ide (right unitor), 
and o: © > ©oS (symmetry) 


subject to the expected coherence axioms [I8]. A (symmetric) monoidal V- 
functor (C, ©1, U1) > (D, ©2, U2) is a lax (symmetric) pseudo-monoid homo- 
morphism, which means that it consists of a V-functor h: C —> D and two 
V-natural transformations h°: Uz > h o U; and h?: @20(h@h) 4 hoe, 
that are coherent with the associators, unitors and symmetries [33]. Together, 
symmetric monoidal V-categories and functors form a category V-SMCat. 

Our goal is now to lift the adjunction between enriched categories from The- 
orem |1|to also include enriched monoidal structure. To this end, we introduce 
lax monoidal strict 2-functors, which are tuples (G, G}, G?) where G: V — W 
is a strict functor of 2-categories and (G, G°, G?): (V, @, I) > (W, x, *) is a lax 
monoidal functor on the underlying 1-categories. 


Theorem 2. Lax monoidal strict 2-functors (G, G°, G?): (V,@,1I) > (W, x, *) 
induce 2-functors PMon(G): PMon(V,®,/) > PMon(W, x,*) between 2- 
categories of pseudo-monoids, lax homomorphisms and 2-cells that are compatible 
with the homomorphism structures. If a G has a monoidal left adjoint F, then 
PMon(F) is left adjoint to PMon(G). Finally, if the monoidal categories and 
functors are symmetric, then the adjunction can be improved to one between 
symmetric pseudo-monoids. 


Proof. The details and appropriate diagram chases are written in Appendix 
A], which go through the following steps. We begin by showing that PMon(G) 
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maps a pseudo-monoid (C,©,U,a, A, p,0) in (V-Cat, &, I) to a pseudo-monoid 
(GC, G(©) o G?, G(U) o G®, Ga, GA, Gp, Go) in (W-Cat, x, *) by checking that 
it fulfills the pseudo-monoid axioms [I8]. 

Similarly, we check that (G,G°,G?) maps a pseudo-monoid homomorphism 
(h, h?, h?) to a pseudo-monoid homomorphism (Gh), G(h°), G(h?)). 

If G has a strict left adjoint F, which is also strong monoidal, we show that 
(F, F°, F?) 4 (G, G}, G?) is a monoidal 2-adjunction if the mates [27] of G° and 
G? are the inverses of F° and F?, respectively, as in the following equations, 
where (4p is the natural isomorphism W(A, GB) => V(FA, B) and 7 the unit 
of the adjunction: 


(F°)! = (G°) and (F?) = (G°) o F(n x n). 


The following theorem, which shows that the change of enrichment extends to 
monoidal enriched categories, follows from Theorem [2] using that V-SMCat = 
PMon(V-Cat, ®,/) and that the change of enrichment gives a lax monoidal 
2-adjunction [717]. 


Theorem 3. If (G,G°,G?): (V,@,I) —> (W,x,*) is a symmetric monoidal 
functor between symmetric monoidal categories with a monoidal left adjoint 
(F, F°, F?), then there are adjunctions that commute with the forgetful functors 
as in the following diagram. 


F, 
V-SMCat © ı | W-SMCat 
Gx 
Fy 
| | 
V-Cat <1, W-Cat 
Gx 


From this theorem and combining the results from Section [3] we can derive 
the following corollary, which is our main tool for building monoidal diagrams 
that are enriched with algebraic operations. 


Corollary 2. A monoidal monad T on Set with an adjunction between the free 
T-algebra functor and the underlying category functor (see Corollary [i) gives 
a free-underlying adjunction (—)o : Alg’-Cat = Cat : F,. This adjunction 
lifts to an adjunction between the 2-categories of symmetric monoidal Alg’- 
enriched and Set-enriched categories (—)o : Alg’-SMCat = SMCat : Fy. 
More explicitly, given such a monad T on Set and a SMC (C,®,I,a,, p,c) 
we can construct the freely Alg” -enriched SMC (C, @,J,a,A, p,a). 


Knowing that we keep the symmetric monoidal structure after doing the free 
enrichment, we can justify drawing parallel composition of probabilistic opera- 
tions in diagram form. Continuing example (3), let us have another probabilistic 
process in which f’ and g’ occur with probabilities 0.7 and 0.3 (respectively) 
parallelly composed. Then we can draw the following picture. 
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0.3 


a 
—— 

A 
a 


T 
7 


In this section, we show an example application of the categorical constructions 
of the previous sections. In particular, we are interested in demonstrating how we 
can take the Distribution monad and enrich the quantum categories of interest 
for reasoning about probabilistic processes in quantum systems. Most impor- 
tantly, we show how the ZX-calculus, a graphical calculus for reasoning about 
quantum processes, can be appropriately extended to accommodate the extra 
structure on said categories and how additional graphical rewrite rules capture 
the interaction of probabilistic and deterministic quantum operations. We begin 
with a general introduction to quantum computing and ZX-calculus, and then 
follow with the enrichment of our categories of interest, together with the intro- 
duction of the extended notation, and we finish by giving an example of how we 
can use this for diagrammatic reasoning of noise in quantum systems. 


men 0 
f9 
y 
h 


5 Applications: ZX-calculus 


5.1 Quantum Computing 


When referring to quantum systems and operations, we have to make a dis- 
tinction whenever we take impure operations into account. In the pure states 
formalisms, quantum states are normalized vectors in a Hilbert space of di- 
mension C?”, with n the number of qubits (quantum bits) of the system. It is 
common to use Dirac bra-ket notation to represent states, for example, some 
important single-qubit states are |0) = [}],|1) = [9], |+) = Fa - (JO) +|1)),|-) = 
Fa - (10) — |1)). We operate on qubits by performing unitary transformations U 
on the quantum states. A multi-qubit quantum system with states |Y} and |) 
corresponds to the tensor (Kronecker) product of the quantum states: |Y) @ |@). 
We will represent the n-fold tensor product of a state |y} by |”). Simultaneous 
(but independent) operations also follow from tensoring unitaries. 

When we take into consideration the possibility of applying non-unitary oper- 
ations we require a more general framework, which is the density matrix and com- 
pletely positive maps formalism. In this case, quantum states are positive semi- 
definite Hermitian matrices p of trace one. We write them as p = )0, pi |Wid(il 
(where (q;| = DAL for ț the conjugate transpose), that is, a statistical en- 
semble of quantum states |~;) (as density matrices) with probability p;. Op- 
erations on density matrices are completely positive (CP) maps of the form 
®: p + >, KipK! with the condition 7, K,K} < 1 (notice how unitary maps 


Enriching Diagrams with Algebraic Operations 131 


fall inside this description too). When we want to reason about quantum systems 
in the presence of noise, we then have to use the density matrix and CP map 
formalism. For more information on quantum computing, we refer the reader 
to [40], and for a more categorical introduction to [20]. 


5.2 The ZX-calculus 


The ZX-calculus is a graphical language for reasoning about quantum states 
and processes as diagrams. The language consists of a set of generators, which 
are the green and re] spiders (also called Z and X spiders), the Hadamard boz, 
the identity wire, the swap, the cup, the cap, and the empty diagram. In[Figure 2] 
we can see the generators of the ZX-calculus and their signature, with input 
wire(s) coming from the top and outputs going to the bottom. Spiders have a 
phase a € [0, 27), which as we will see later is omitted when a = 0. We can also 
see how to sequentially compose (o) arbitrary diagrams by connecting inputs 
with outputs, and how to parallely compose diagrams (as a tensor product @) 
by placing them side by side. 


m ZX-diagrams 


Fig. 2: ZX-diagrams generators and how to compose them. 


Each of the generators has a standard interpretation |-| as a linear map in 
C?” that we can find in [Figure 3] 

Categorically, ZX-diagrams form the category ZX with |ZX| = N (where 
some n € N is the number of wires, which we can think of as an n-qubit quantum 
system) and morphisms being the generators. The standard interpretation is a 
(monoidal) functor [-] : ZX — Qubit that acts on objects as [n] = n and on 
morphisms as defined in [53]. 

ZX-diagrams come with a set of rewrite rules that form the ZX-calculus. 
These rewrite rules let us transform a diagram into a different one while pre- 
serving the semantics (i.e. the interpretation). We have collected the rules in 


3 Light and dark in grayscale, respectively. 
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m Standard interpretation 


n 


LI 


LEN 100] + et] Ul, [HON] 4 eta (my (_(m)| 


Dı 


T, [De] o [D] Dı Də | 4, [di] @ [P] 


D2 


Fig. 3: Standard interpretation of ZX-diagrams. 


Figure 4| There is also an important additional rule that can be summarized as 
the only connectivity matters rule, which states that we can deform diagrams 
at will without changing their meaning, as long as we maintain the connectivity 


between the generators unchanged. For a thorough explanation of each rule we 
refer the reader to [15152]. 


The ZX-calculus satisfies important properties. ZX-diagrams are universal, 
meaning that any linear map f of the form f : C?” — C2?” can be represented 
as a ZX-diagram. The rewrite rules are sound, meaning that they do not change 
the interpretation of the diagram as a linear map. They are also complete, which 
ensures that if two diagrams have the same interpretation, the ruleset is powerful 
enough to always let us transform one diagram into the other. These properties 
ensure that the ZX-calculus can be used as a tool for reasoning about quantum 
computing, as it has been already demonstrated in tasks such as quantum circuit 
optimisation [30], verification of quantum circuits MI], simulation [SI], and as a 
reasoning tool [4]29]. 


In we have example one- and two-qubit gates as ZX-diagrams. We also 
see the computational basis {|0),|1)} and Hadamard basis {|+),|—)} states. 
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ZX-calculus ruleset 


Fig. 4: ZX-calculus ruleset. All rules also hold when swapping the colors of the 
spiders. In (eu) we omit the calculation of the angles, which can be found in [52]. 


1000 100 0 
0100 „_ loroo], 
CNOT = lygia -o = lo01 0| = aay 
0010 000-1 
=a? n= d IH) =H 9 BREA 


5.3 Enriching the Categories Qubit and CPM(Qubit) 


Our motivation is to highlight certain types of relevant physical phenomena 
(probabilistic processes) that are present in quantum systems within our cat- 
egories. It is then natural to use the Distribution monad D together with the 
construction explained in the previous sections to enrich our categories for quan- 
tum reasoning. 

Indeed, we take CPM(Qubit) and perform a free enrichment over D. What 
we get is the category (Ff, CPM (Qubit), @, J) consisting of the same objects as 
CPM(Qubit) and morphisms (incl. identity) for objects A, B the free algebras 
over D of the hom-set CPM(Qubit)(A, B). Composites of morphisms are the 
free algebra over the composite in CPM(Qubit), and the SMC structure is 
preserved thanks to Corollary [2] 

We also define the non-freely enriched category CPM(Qubit) so we can in- 
terpret probability distributions as CP maps. For this, we define SMC-structure 
in the non-free (Alg?, @?,D(«)), with a tensor product of algebras defined by 
the coequalizer (i), with a more detailed description in Appendix B]. The 
category (CPM(Qubit),©,U) has the same objects as CPM(Qubit) and for 
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every pair of objects A, B the hom-object is an algebra (CPM(Qubit)(A, B), a) 
with the D-action turning a formal convex sum of linear maps into an actual 
sum by scalar multiplication and addition. Composition of hom-objects follows 
from composition in CPM(Qubit), and for an object A the identity element 
is ja : (x,a) > (Id4,a). We define now its symmetric monoidal structure fol- 
lowing the definition of enriched SMC from the beginning of Section |4| The 
tensor product © on objects is the same as in CPM(Qubit), and on hom- 
objects it is the tensor product in CPM(Qubit) to the underlying sets: © : 
(CPM(Qubit) (A, A’), v) @? (CPM(Qubit)(B, B’), av) + (CPM(Qubit)(A® 
B, A'® B’),a). The unit U is the one in CPM(Qubit). The associator, unitors, 
and symmetry all follow from applying the ones in CPM(Qubit). 

In the following sections, we will interpret ZX-diagrams into F,CPM(Qubit) 
as probability distributions of CP maps. From there, to interpret probability 
distributions as CP maps, we define the functor ((:)) : F.CPM(Qubit) —> 
CPM(Qubit) that sends objects to themselves and applies the monad alge- 
bra to hom-objects i.e. we “evaluate” a probability distribution over CP maps 
by multiplying the probabilities with the corresponding map and then adding 
all maps together. 

Technically, we can also enrich Qubit in the same way as we did with 
CPM(Qubit), but density matrices and CP maps are the more sensible choices 
to talk about probabilistic mixtures of operations. On the other hand, enriching 
Qubit (or CPM(Qubit)) over algebras of the multiset monad M leads to an 
enrichment over commutative monoids that exposes addition of linear maps [20]. 
This was recently formulated in [39/50] as a way to “split” parameterised Pauli 
rotation gates in ZX-calculus in such a way that the parameter relocates from 
its place inside the spider as a phase to a scalar on a wire using the identity 
e’°P = cosal +isinaP for P a Pauli matrix (or any matrix satisfying P? = I). 


5.4 Enriched ZX-diagrams and Their Interpretation 


In the same manner as the ZX-calculus is a language for reasoning in Qubit, we 
can create a graphical language with extra structure to reason in our enriched 
categories. Since we are going to be enriching CPM(Qubit), we first need to 
see how to turn the ZX-calculus into a graphical language for CP maps. This is 
done straightforwardly by adding a discard operation + to the list of generators 
of plus additional rewrite rules (that we choose to omit here) stating 
that isometries can be discarded [I3]. The interpretation of a ZX-diagram D 
as a CP map is then a superoperator p > [D] p [D]', for [D] the standard 
interpretation of D as in [Figure 3] [9]. 

We then construct an enriched graphical language for CPM(Qubit) by 
building on top of the ZX-calculus for CP maps. The notation will be simi- 
lar to the running examples we have given throughout the text (cf. (BF). The 
main idea is as follows. We take the generators of the ZX-calculus and allow 


them to be freely wrapped between opening H and closing W distribution 


brackets. 
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Intuitively, we interpret diagrams that are within distribution brackets as a 
probabilistic mixture of operations: diagrams placed side by side correspond to 
different probabilistic choices with some weight attached to the corresponding 
wires. Within each choice, sequential (and as we will see later, parallel) composi- 
tion is allowed. The main difference to the usual graphical languages for monoidal 
categories is that the parallel composition of each choice does not correspond to 
the tensor product. In a way, we also subsume ZX-diagrams by drawing diagrams 
that are not enclosed by distribution brackets, which are then interpreted as an 
operation that occurs with probability 1. 

For example, we can represent the single-qubit depolarizing channel [40] @: 
p> (1—p)pt+8(XpX +Y pY + ZpZ) that leaves a quantum state p unchanged 
with probability (1 — p) or applies an X,Y or Z error with probability 4 each 
with the diagram on the left in Figure 


tors 


Fig. 5: Left: Diagrammatic representation of the depolarizing channel. 
Right: Diagrammatic representation of a mixture of two-qubit gates. 


We need to take extra care when handling scalars inside the brackets. Indeed, 
what we have inside distribution brackets is a formal convex sum of ZX-diagrams 
(or, in the general case, string diagrams), meaning that the SMC rewriting ax- 
ioms apply to each summand independently. Since summands are also juxta- 
posed, it might seem like this notation allows for the transfer of scalars from one 
summand to another. The crux is that, since what is enclosed by trapezoids is a 
formal sum, we cannot drag scalars from one summand to another using those 
same monoidal category axioms. This means that we can consider the probabil- 
ities (and any scalar factor if present, such as the imaginary unit in Figure |5) to 
be bound to the wires themselves, and only interact with the ZX-diagrams (or 
generally string diagrams) that belong to that summand. An alternative is to 
encapsulate each summand in “bubbles” for stronger visual separation [50]39]. 

A probabilistic mixture of operations with multiple inputs or outputs looks 
similar to the 1-to-1 case, with the caveat that we need to be more careful 
in the positioning of the wires as to distinguish between tensor product and 
probabilistic choice] For example, if we want to represent applying the CNOT 


t One could use scalable notation to allow wires to be multi-qubit quantum reg- 
isters. This would help with the distinction when diagrams are larger in practice. 
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gate and the CZ gate with probabilities p and 1 — p respectively we would get 
the diagram on the right in Figure 

We can consider this extra notation as the result of a free enrichment of ZX 
over Alg? giving us the category of enriched ZX-diagrams F,ZX. We can then 
define the interpretation |-]p of an Alg?-enriched diagram as a monoidal func- 
tor from the category of enriched ZX-diagrams to CPM(Qubit). This functor 
factors through F,,CPM(Qubit) as follows: 


F.2X Č? F.CPM(Qubit) 


mœ~ WO 


CPM(Qubit) 


Where (-)), interprets an enriched ZX-diagram as a probabilistic mixture 
of operations which is then evaluated by ((-)) as explained in Section An 
example of the interpretation of an arbitrary distribution of ZX-diagrams of 
arbitrary size can be seen in (6). When using the multiset monad M instead the 
interpretation [-] is similar. 


Pie We 
Dil [De] BP. pus dy pil p (Did (6) 


Enriched ZX-diagrams are universal, that is, any morphism in CPM(Qubit) 
can be represented by an enriched ZX-diagram. Indeed, since CPM(Qubit) is 
still made of CP maps between Hilbert spaces, we can use universality of the 
ZX-calculus alone to represent any morphism in CPM(Qubit). 


5.5 Additional Rules for Enriched ZX-diagrams 


With the new notation we can have new rewrite rules too, some of which were 
already introduced in [50)39] ((es), (ep), (ec), and (e6)) for the case of linear 
combinations. We will display them here, including additional rules. The ruleset 
of the enriched ZX-calculus for the distribution and multiset monads is the same 
as the one for ZX-calculus plus additional rules that capture the interaction 
between sums, products, tensor products, and scalars. We can see the additional 
rules arising from the enrichment in Figure |6| which intuitively state: 


— (es): The enriched sequential composition rule shows how to sequentially 
compose distributions. Intuitively this rule follows from products distributing 
over addition. 

— (ep): The enriched parallel composition rule is the same as (es), but for 
parallel composition instead of sequential. 
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Additional enriched ruleset 


n+r 


Pig Prqe 


m+s 


alts Bu w 
M n Pı p2 T: + p2 A nU ts 
bas fa, DN / 


D| © |p D D (e+) (e0) 


ALA Wd 
Do) {Di g sb 
mm mm mm 


Fig. 6: Additional rules for the enriched ZX-calculus, alongside the ones of Fig- 
ure [4] Diagrams D, D’ are arbitrary ZX-diagrams and weights p,q are probabil- 
ities. 


— (ec): The enriched commutativity rule shows that bracketed diagrams are 
invariant under permutation of the branches. 

— (ed): The enriched Dirac delta distribution rule provides a shorthand for the 
trivial Dirac delta distribution. 

— (e+): The enriched addition rule shows that we can remove a branch if it is 
identical to some other by adding the probabilities. 

— (e0): The enriched 0-probability rule allows us to remove branches with 0 
probability . 


Rules (es),(ep),(ec) and (ed) were proven to be sound in [89] but in the 
context of linear combinations of diagrams interpreted in Qubit. We show that 
these rules still hold as an enrichment in Alg? and interpreted in CPM(Qubit) 
in [BI] Appendix C]. Finding a complete ruleset (i.e. one that can show Dı = D2 
whenever [D,]p = [D2]p) for enriched diagrams remains to be done. A possible 
direction to tackle this problem would be to translate enriched diagrams into 
ZXW [49] diagrams, which is a complete diagrammatic language with a W-spider 
that can encode addition of phases. Another alternative would be to translate 
into the controlled form of [25]. 

We conclude with a demonstration of how we can use this extension of the 
ZX-calculus to study the effectiveness of Quantum Error Mitigation (QEM) tech- 
niques for different noise models. Quantum Error Mitigation are the series 
of techniques that are used to reduce the effects of noise in near-term quan- 
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tum systems. One such technique is Symmetry Verification [6], which states 
that given a Hamiltonian (Hermitian operator that determines the evolution 
of a system) H, and a symmetry S (an operator that commutes with H i.e. 
[A ,S] = HS — SH = 0), one can perform measurements of S to verify if the 
state that (ideally) evolves under H was affected by errors. Indeed, under the 
assumption that the initial state is a (+1) eigenvector of S, then it will stay 
that way under ideal evolution under H. This implies that if there is an error E 
that anti-commutes with S (i.e. {E, S} = ES + SE = 0) at some point in the 
computation, we can measure S to detect a change in the eigenvalue. Symmetry 
verification then proposes to perform a postselection on the result (+1), meaning 
that we discard computations that give a (—1) outcome when measuring S. 

Given a noisy state Ppoisy and a symmetry S, the probability of outcome 
(+1) when measuring $ is given by p(+1) = tr(Py1Pnoisy), for Py: = #2 the 
projector onto the (+1) eigenspace of S and tr the trace operator. This value 
tells us then with which probability the measurement “accepts” a noisy state, and 
can be used to compare the effectiveness of different choices of S' given a certain 
noise model [26]. Let us consider Pyoisy = &(U |0)) for @ the depolarizing noise 
channel and U some single-qubit unitary — in other words, we have a single layer 
of depolarizing noise at the end of our computation. For simplicity, let us further 
assume that our state before the depolarizing channel is the (+1) eigenvector of 
some Pauli operator e.g. the o X, then we have U = H (the Hadamard gate) 
and we can draw p(+1) = tr(42 Bi sins) diagrammatically (up to scalar factor, 
see [51] Appendix D]) as the following diagram: 


From top to bottom, the diagram represents applying H to the |0) state, 
followed by a depolarizing noise channel and the verification of X in the form of 
a CNOT gate controlled on an auxiliary qubit. The auxiliary qubit on the right 
starts in the |0) state and has a Hadamard gate applied to it before and after the 
CNOT. It is then postselected into (0|, which is the corresponding state for the 
(+1) outcome. The last operation in the form of + corresponds to the trace. The 
full diagrammatic calculation is in [5I] Appendix D]. With similar diagrams, we 
can study diagrammatically how well different QEM techniques mitigate certain 
noise models, and apply them to representations of quantum algorithms that, 
for example, have one layer of errors for every time step. 
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6 Discussion and Future Work 


In this work, we have shown how to construct freely enriched symmetric monoidal 
categories over the algebras of a monoidal monad on Set that satisfies F 4 
Alg’ (I,—) for F the free T-algebra functor and I the unit of the monoidal 
structure, which is the case in particular when T is an affine monad. We have 
then taken this construction and developed a graphical language that captures 
the additional algebraic structure of the morphisms for the case of the Distribu- 
tion monad. We then show how we can use this to study classical probabilistic 
processes in quantum systems, a highly relevant type of operation for near-term 
quantum applications. In particular, we extend the ZX-calculus to make it a 
language for reasoning in an enriched version of CPM(Qubit). 

We believe that this work opens several directions for future research. The 
most evident one is to prove completeness of the enriched diagrams, which in 
turn would facilitate automated implementations for tasks such as simulation of 
noisy quantum systems, fine-tuned quantum circuit optimization techniques for 
specific quantum devices, or comparison of the effectiveness of different Quantum 
Error Mitigation techniques. An interesting venue would be to use enrichment 
over M to reason about quantum circuit pre- and post-processing techniques, 
such as circuit cutting [44]42], in which quantum circuits are “split” into linear 
combinations of smaller ones that are executed separately. We also believe that it 
could be possible to integrate monads that capture quantum behaviours into our 
construction to represent in enriched ZX-diagrams phenomena such as superpo- 
sition of execution orders, like what is done in the Many-Worlds calculus [14]. 

Strongly related to completeness is to have presentations of the diagrams in 
terms of generators and equations. We achieved this by hand in Section by using 
that the algebras for the distribution monad can be presented as convex algebras 
with a family of operations +p. The question is then what the analogue of convex 
monads is when using algebras presented by Lawvere theories or sketches [2136]. 

We are also interested in finding other monads that could capture interesting 
processes outside of the quantum realm. For example, the non-empty powerset 
monad could be used to encode non-deterministic operations and be used for 
reasoning about a third party operating on a shared quantum system. 
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Abstract. Extensions of Stone-type dualities have a long history in 
algebraic logic and have also been instrumental for proving results in 
algebraic language theory. We show how to extend abstract categorical 
dualities via monoidal adjunctions, subsuming various incarnations of 
classical extended Stone and Priestley duality as a special case. Guided by 
these categorical foundations, we investigate residuation algebras, which 
are algebraic models of language derivatives, and show the subcategory 
of derivation algebras to be dually equivalent to the category of profinite 
ordered monoids, restricting to a duality between boolean residuation 
algebras and profinite monoids. We further extend this duality to capture 
relational morphisms of profinite ordered monoids, which dualize to 
natural morphisms of residuation algebras. 


Keywords: Stone Duality - Profinite Monoids - Regular Languages. 
1 Introduction 


Marshall H. Stone’s representation theorem for boolean algebras, the foundation 
for the so called Stone duality between boolean algebras and Stone spaces, 
manifests a tight connection between logic and topology. It has thus become an 
ubiquitous tool in various areas of theoretical computer science, not only in logic, 
but also for example in domain theory and automata theory. 

From algebraic logic arose the need for extending Stone duality to capture 
boolean algebras equipped with additional operators (modelling quantifiers or 
modalities). Originating in Jonsson and Tarski’s representation theorem for 
boolean algebras with operators [21,22], a representation in the spirit of Stone 
was proven by Halmos [17]; the general categorical picture of the duality of Kripke 
frames and modal algebras is based on an adjunction between operators and 
continuous relations developed by Sambin and Vaccaro [31]. 

In the study of regular languages, the need for extensions of Stone duality 
was not discovered until this millenium: while Pippenger [27] has already shown 
that the boolean algebra of regular languages on an alphabet X corresponds, 
under Stone duality, to the Stone space X* of profinite words, Gehrke et al. [15] 
discovered that, under Goldblatt’s form of extended Priestley duality [16], the 
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residuals of language concatenation dualize to multiplication on the space of 
profinite words. But while categorical frameworks have identified Stone-type 
dualities to be one of the cornerstones of algebraic language theory [36,30], the 
correspondence between residuals and multiplication via extended duality has 
not yet been placed in the categorical big picture. One reason is that, despite 
some progress in recent years [6,18], extended (Stone) dualities for (co-)algebras 
are themselves not fully understood as instances of a crisp categorical idea. 

Therefore we introduce as our first main contribution a simple, yet powerful 
framework to extend any categorical duality C ~°P Ĉ via monoidal adjunctions: 
For a given adjunction on C with a strong monoidal right adjoint U we prove a 
dual equivalence between the category of U-operators on C to dual operators 
in the Kleisli category of the monad on C arising from the dual of the given 
adjunction. We show how to instantiate the abstract extended duality to Priest- 
ley duality, which not only recovers Goldblatt’s original duality for distributive 
lattices with operators [16] but also applies more generally to bialgebraic op- 
erators with relational morphisms. Guided by our categorical foundations for 
extended Stone duality we investigate the correspondence between language 
derivatives and multiplication of profinite words in the setting of residuation 
algebras originally studied by Gehrke [14]. The key observation is that on finite 
distributive lattices the residuals are equivalent to a coalgebraic operator on the 
lattice, and we show how to lift this correspondence to locally finite structures, 
i.e. structures built up from finite substructures. By identifying suitable non-full 
subcategories — derivation algebras and locally finite comonoids, respectively — 
and an appropriate definition of morphism for residuation algebras, we augment 
Gehrke’s characterization of Stone-topological algebras in terms of residuation 
algebras to a duality between the categories of derivation algebras and that of 
profinite ordered monoids: 


Der ~ Comon; ~°? ProfOrdMon. (1.1) 


The above duality clarifies the relation between Gehrke’s results and the duality 
by Rhodes and Steinberg [29] between profinite monoids and counital boolean 
bialgebras. The extended duality now suggests that the dual equivalence between 
profinite ordered monoids on one side and locally finite comonoids as well as 
derivation algebras on the other side extends to a more general duality capturing 
morphisms of relational type of profinite ordered monoids. To this end, we identify 
a natural notion of relational morphism for residuation algebras and comonoids, 
and use our abstract extended duality theorem to obtain the dual equivalence 


RelDer = RelComon; ~°? RelProfOrdMon 


which extends (1.1) to relational morphisms. To our knowledge, this is the first 
duality result for relational morphisms of profinite monoids, which have become 
an ubiquitous tool in algebraic language theory [26] and semigroup theory [29]. 
Full proof details can be found in the full version [5] of this paper. 


Related Work. Duality for (complete) boolean algebras with operators goes back 
to Jonsson and Tarski [21,22]. This duality was refined by the topological approach 
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via Stone spaces taken by Halmos |17], which allowed to characterize the relations 
arising as the duals of operators, namely boolean relations. Halmos’ duality 
was extended to distributive lattices with (n-ary) operators by Goldblatt [16] 
and Cignoli [7]. Kupke et al. [24] recognized that boolean relations elegantly 
describe descriptive frames as coalgebras for the (underlying functor of) the 
Vietoris monad on Stone spaces; notions of bisimulation for these coalgebras were 
investigated by Bezhanishvili et al. [2]. Bosangue et al. [6] introduced a framework 
for dualities over distributive lattices equipped with a theory of operators for 
a signature, which are dual to certain coalgebras. Hofmann and Nora [18] have 
taken a categorical approach to extend natural dualities to algebras for a signature 
equipped with unary operators preserving only some of the operations prescribed 
by the signature; they relate these to coalgebras for (the underlying functor of) a 
suitable monad T. In their framework T is a parameter required to satisfy certain 
conditions for the duality to work, while in our work T is already determined by 
the adjunction. The recent work by Bezhanishvili et al. [1] clarifies the relation 
between free constructions on distributive lattices and the different versions of 
the Vietoris monad to derive several dualities between distributive lattices with 
different types of operators and their corresponding Priestley relations. 

Residuated boolean algebras, i.e. boolean algebras with a residuated binary 
operator,were explicitly considered by Jónsson and Tsinakis [23] to highlight 
the roles of the residuals in relation algebra. Gehrke et al. [15] discovered the 
connection between the residuals of the concatenation of regular languages and 
the multiplication on profinite words and investigated applications to automata 
theory, most notably a duality-theoretic proof of Eilenberg’s variety theorem [8]. 
The duality theory behind the correspondence of general residuation algebras 
and Priestley-topological algebras was given via canonical extensions [12,11] and 
Goldblatt’s extended Stone duality [16] by Gehrke [14]. She has also provided 
conditions under which the dual relations of the residuals is functional; Fussner 
and Palmigiano [10] have shown that functionality of the dual relation is not 
equationally definable in the language of residuation algebras. 


2 Preliminaries 


Readers are assumed to be familiar with basic category theory, such as functors, 
natural transformations, adjunctions and monoidal categories [25]. We briefly 
recall the foundations of Stone duality [34] and Priestley duality [28]. By the 
latter we mean the dual equivalence DL ~°? Priest between the category DL 
of bounded distributive lattices and lattice homomorphisms, and the category 
Priest of Priestley spaces (ordered compact topological spaces in which for every 
x Å y there exists a clopen up-set containing x but not y) and continuous mono- 
tone maps. The duality sends a distributive lattice D to the pointwise-ordered 
space DL(D, 2) of homomorphisms into the two-element lattice (equivalently 
prime filters, ordered by inclusion), and topologized via pointwise convergence. 
In the reverse direction, it sends a Priestley space X to the distributive lat- 
tice Priest (X,2) of continuous maps into the two-element poset 2 = {0 < 1} 
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with discrete topology (equivalently clopen upsets), with the pointwise lattice 
structure. Priestley duality restricts to Stone duality BA ~°P Stone between 
the full subcategories BA of boolean algebras and Stone of Stone spaces (dis- 
cretely ordered Priestley spaces). Moreover, it restricts to Birkhoff duality [3] 
DL; ~ Pos; between finite distributive lattices and finite posets, sending 
a finite distributive lattice to its poset of join-irreducibles and a poset to its 
lattice of upsets — note that the pointwise order on homorphisms induces the 
reverse order on join-irreducibles. For a comprehensive introduction to ordered 
structures and their dualities, see the first two chapters of the classic textbook 
by Johnstone [20]. 


3 Extending Dualities 


We present the first contribution of our paper, a general categorical framework 
for extending Stone-type dualities via monoidal adjunctions, motivated by the 
extension of Priestley duality to operators due to Goldblatt [16] recovered in 
Section 4. It serves as the basis for our duality results in the next two sections. 


Notation 3.1. (1) For U: C > D being right adjoint to F: D — C we write 
F:DiAC:U orsimply F 4U. We denote the unit and counit by 7 and € 
and the transposing isomorphisms by 

(—)+: D(C,UD) S C(FC, D) :(-)~ with f Se Ff, g =Ug-n. 


(2) For dually equivalent categories C and C we denote the equivalence functors 
in both directions by (~): C = G and (~): © = C. Moreover, if if F: C + D 
is a functor and Ô is dual to D, we denote its dual by F = (~)o Fo (2): C > D. 
(3) The Kleisli category of a monad (7,7, u) on C is denoted by Cr. It has 
the same objects as C and Cr(X,Y) = C(X,TY) with Kleisli composition 
gof =pu-Tg-f. A morphism f: C > TD of the Kleisli category is pure if 
f =n; f for some f’: C > D in C. (We omit the components of 7 and p.) 


Assumptions 3.2. We fix monoidal categories C,D with dually equivalent 
categories C,D; we regard Ĉ, Ô as monoidal categories with tensor products 
® dual to the tensor products ® of C,D. Moreover, we fix an adjunction 
F:DAC:U with unit 7: Id > UF and counit e: FU — Id, and assume that U 
is a strong monoidal functor with associated natural isomorphisms à: UX ® 
UY = U(X @Y) and e: Ip = Ulg. One can extend to an isomorphism 
à: QL, UX; = U(@"™, X;) for all finite n. The dual functor U: C > D is a 
strong monoidal left adjoint to F and the unit and counit of this dual adjunction 
are € and 7. We denote the monad dual to the comonad FU by T = FU with 
unit e = ê: Id > T and multiplication m = FAU: TT >T. 


D ~ D 


Oo 


C ~oP C DOT 
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Remark 3.3. Since U is strong monoidal with €: Ip S Uig and å: UX®UY = 
U(X®Y) its right adjoint F is (lax) monoidal (see e.g. [32, p. 17]) with 


(ASÀ) -A71)—: FX@FY —> F(X®Y) and (6&1): Îo > Fip. 


This makes U 4 F a monoidal adjunction, which then induces a monoidal monad 

= FU on Ĉ. Let 6: TX@TY > T(X@Y) denote the witnessing natural 
transformation, which also extends to any arity. The tensor product © of C lifts 
to the Kleisli category Cr; the lifting sends a pair (f: X > TY, g: X’ > TY’) of 
Cr- morphisms to the Cr-morphism ô- (fg): XX’ > TY@TY’ > T(Y@Y’). 
This makes Cr itself a monoidal category [33, Prop. 1.2.2] with tensor ® and 
the canonical left adjoint Jr: Cs Cy a strict monoidal functor. 


Definition 3.4. Let G: A — B be a functor between monoidal categories, and 
let m,n € N. An (m,n)-ary G-operator consists of an object A € A and a 
morphism a: (GA)®” — (GA)®” of B. An (m,n)-ary G-operator morphism 
from (A, 6) to (B,b) is a morphism h: GA > GB of B such that 


(GA)®™ ae (GA)®” 


nom | [ae 


(GB)@” —+ (GB)®” 


commutes. The category of (m,n)-ary G-operators is denoted by Opé’"(A). We 
call (m, 1)-ary G-operators G-algebras and (1,n)-ary G-operators G'-coalgebras. 
If G is strong monoidal we call an operator pure if it is of the form A~!- Ga’ - À, 


for \ analogous to Assumptions 3.2, and an operator morphism pure if it is of 
the form Gh’. 


Note that the full subcategory of B consisting of the objects in the image of 
G fully embeds into Opg (A) via GA (GA, idea). 


Theorem 3.5 (Abstract Extended Duality). The category of (m,n)-ary 
U-operators is dually equivalent to the category of (n,m)-ary Jr-operators: 


Opp” (C) =°? Oph" (Ô). 


Proof (Sketch). The functor Op” (Ô) — Opp” (C) is defined as follows. An ob- 


ject of Op” (Ô) is an operator â: A®" + TÂÊ™, By dualization, transposition 
and conjugation with A it is mapped to 


Away A: (UA)O™ SU AS™ > UAS” = (UV A)?” 
An operator morphism f: (A, a) > (B, b) is mapped to f—: UB > UA, the dual 


of its transpose; a diagram chase shows that this is indeed an operator morphism. 
One then proves that this yields a dual equivalence 
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An advantage of extending dualities via adjunctions is that adjunctions 
compose, making the extensions modular: let E be a monoidal category with 
monoidal adjunctions Fi: E 4 C :U; and Fp: DIE :Uə splitting F 4 UV, i.e., 
F = FF, and U = UgU, and A = U2A1-A2QU,. Then the following lifting property 
applies to operators (set A = B) as well as operator morphisms (set m =n = 1): 


Proposition 3.6. A morphism a: (UA)®™ — (U B)®” in D lifts to a morphism 
b: (U, A)®™ — (U, B)®” with a = A3" - U2b- Az iff the dual of a factors through 
the canonical monad morphism Ê ê: Tı > T, where T, = Ê Ô. 


Remark 3.7. (1) A special case of Proposition 3.6 proves that extended dualities 
preserve purity: splitting F 4 U into Fı = Id 4 Id =U, and Fp =F AU = U2 
we see that a U-operator (or operator morphism) a is pure iff its dual f is pure 
as a Kleisli morphism, i.e. factors through the unit e of T. 


(2) The right adjoint U2 often is faithful and in this case F\éU, is monic, i.e. T; 
is a submonad of T: faithfulness of Uz is equivalent to having an epic counit €2, 
hence ĉ2Û; is mono, and the right adjoint F, preserves monos. In particular, if T 
is “powerset-like”, then Cr isa category of relations, and we think of U-operators 
(or operator morphisms) of the form a = Az! - Uzb- Az as dualizing to “more 
functional” relations. The examples of Section 4.2 illustrate this idea. 


4 Example: Extended Priestley Duality 


As a first application of our adjoint framework, we investigate the classical 
Priestley duality (Section 2) and derive a generalized version of Goldblatt’s 
duality [16] between distributive lattices with operators and relational Priestley 
spaces. We instantiate (3.1) to the following categories and functors, which we 
will subsequently explain in detail: 


D ~ D JSL ~P StoneJSL 
oi Aer © or DL ~eop Priest a: 


Categories The upper duality is Hofman-Mislove-Stralka duality [19] between the 
category of join-semilattices with bottom and the category of Stone semilattices 
(i.e. topological join-semilattices with bottom whose underlying topological space 
is a Stone space) and continuous semilattice homomorphisms. The duality maps 
a join-semilattice J to the Stone semilattice JSL(J, 2) of semilattice homomor- 
phisms into the two-element semilattice, topologized by pointwise convergence. 
Equivalently, JSL(J, 2) is the space Idl(J) of ideals (downwards closed and up- 
wards directed subsets) of J, ordered by reverse inclusion, with topology generated 
by the subbasic open sets o(j) = {I € Idl(J) | j € I} and their complements for 
j € J. In the other direction, a Stone semilattice X is mapped to its semilattice 
StoneJSL(X, 2) of clopen ideals, ordered by inclusion. 
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Functors The functor U: DL —> JSL is the obvious forgetful functor. Its left 
adjoint F: JSL + DL maps a join-semilattice to the set Ug (J) of finitely 
generated upsets of J ordered by reverse inclusion. The dual right adjoint Ê 
of the left adjoint F is the forgetful functor mapping a Stone semilattice to its 
underlying Priestley space. Indeed, as U2 = 2 we compute for the underlying 
Priestley space |X| of a Stone semilattice X that 


FX = DL(F(StoneJSL(X, 2)), 2) = |JSL(StoneJSL(X, 2), U2)| = |X], 


and this bijection is a homeomorphism. Its left adjoint Û: Priest — StoneJSL 
maps a Priestley space X to the space 


UX = JSL(U(Priest(X,2)),2) = Idl(Cl; X) = V, X 


of ideals of clopen upsets of X. This space is isomorphic to the (downset) Vietoris 
hyperspace V| X of X that has as carrier the set of closed downsets of X. The 
isomorphism Idl(Cl, X) = V, X maps an ideal J to the intersection []yez X \ U; 
its inverse sends a closed downset C to the ideal {U € Cl X | C C X \ U} 
of complements of the basic clopen downsets that contain it. The topology of 
pointwise convergence on JSL(U(Priest(X, 2)),2) translates to the hit-or-miss 
topology on V| X generated by the subbasic open sets 


{AC X closed | ANU #0} for UEChX 


and their complements. For a detailed exposition of these results we refer the 
reader to the recent work by Bezhanishvili et al. [1]; the free join-semilattice 
structure on Vj X was already observed by Johnstone [20, Sec. 4.8]. The unit 
e: X — V, X of the Vietoris monad is given by x +> |x and multiplication is 
given by union [18]. The monad V, restricts to the full subcategory Stone of 
Stone spaces. We denote the restriction of this monad simply by V. 


Remark 4.1 (Continuous Relations). Continuous maps in Priest of the 
form p: X — VY have a variety of names, we use the term Priestley relation as 
in [7,16] or Stone relation if X,Y are Stone spaces. We write x py for y € p(x), 
and sometimes identify p with a subset of X x Y. Let us note that some authors 
(e.g. [29]) call a relation R C X x Y between topological spaces continuous if it 
is closed as a subspace of X x Y. Every Priestley relation is continuous, but a 
continuous relation between Priestley spaces is generally not a Priestley relation. 


Monoidal Structure The category JSL of join-semilattices has a tensor product 
® with the universal property that it extends join-bilinear maps: 


Bilin(J x J’, K) & JSL(J @ J’, K). 


Join-bilinear maps J x J’ > K and their corresponding JSL-morphisms J @ J’ + 
K are often tacitly identified. The tensor product ® makes JSL a monoidal 
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category with unit 2, i.e. 2@ J S J. The tensor product has a representation by 
the generators {j @ 7’ | 7 € J,j’ € J’} and relations 


720 =08j7’ =0, (fiVj2)@k = fi @kVjo@k and j@(kiVk2) = J@kiVj@ke. 


We call generating elements j ® j’ pure tensors. If D, D’ are bounded distributive 
lattices then so is UD@UD" |9], with meet given on pure tensors as (d®d’) A(e@ 
e’) = (dAe) @(d' Ae’). The lattice UDU D' moreover is the coproduct of D, D’ in 
DL: the coproduct injections are (d) = d@1’ and c'(d') = 1@d' ford € D, d' € D’, 
and the copairing of lattice homomorphisms f: D => E, f’: D' —> E is given by 
the extension of the join-bilinar map 


A-(f x f): DxD >E, (d, d') > f(d) A f(d). 


Taking coproducts yields a monoidal structure on DL and since U (D + D’) = 
UD &UD' the functor U is strong monoidal. The dual monoidal structure on 
Priest takes binary products, and the natural transformation ô of Remark 3.3 is 
the expected product of sets 


ô: VX x VY > V(X xY), (C,D)= Cx D. 
Spelling out Definition 3.4, the category Op,” (Priest) is given as follows: 


Definition 4.2. A ((n,m)-ary) relational Priestley space consists of a carrier 
Priestley space X and a Priestley relation p: X” > V, X™. A relational morphism 
from a relational Priestley space (X, p) to (X’, p’) is given by a Priestley relation 
B: X > VY such that, for all x € X”, y € X™,y’ € X™, 


x py (Vi: yi by) > 3x: (Vi: a; Bai)ax' ply’, 
and, for all x € X",x’ e X/,y’ e X”, 


(Vi: xi B £i) Ax pP y'> 3y:xpy (Vi: yi B yi). 
We let Opg” (Priest) denote the category of (n,m)-ary relational Priestley 
1 


operators and relational morphisms. 
Then Theorem 3.5 instantiates to the following result: 


Theorem 4.3 (Extended Priestley duality). The category of (m,n)-ary U- 
operators of distributive lattices is dually equivalent to the category of (n,m)-ary 
relational Priestley spaces and relational morphisms: 


Opp” (DL) >°? Opiy (Priest). 


By taking n = 1 and restricting the operator morphisms to be pure, we 
recover Goldblatt’s duality [16]. Here, pure relational morphisms are called 
bounded morphisms and n-ary U-algebras (UD)®" — UD in JSL are called 
n-ary join-hemimorphisms. 


Corollary 4.4 (Goldblatt, 1989). The category of distributive lattices with n- 
ary join-hemimorphisms, and pure morphisms between them, is dually equivalent 
to the category of (1,n)-relational Priestley spaces and bounded morphisms. 
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4.1 Deriving Concrete Formulas 


We proceed to show how an enriched extension of our adjoint framework can be 
used to methodically derive concrete (i.e. element-based) formulas for the dual 
join operator of a continuous relation and vice versa. Let us first observe that all 
involved categories are order-enriched, i.e. the homsets are (pointwise) partially 
ordered; for JSL and DL this is clear and relations X — V,Y are ordered by 
inclusion, as usual. Moreover, from the definitions it is clear that the transposing 
isomorphisms of the adjunction F 4 U and the duality DL ~ Priest are 
order-isomorphisms. 

Second, in Priest we can represent an element ĉ of a space X as a continuous 
function 1 + X that we also denote by ĉ; on the lattice side, elements of a 
join-semilattice J correspond bijectively to JSL-morphisms 2 > J. 

For the rest of the section we fix a U-algebra h: (UX)®”" + UX with dual 
Priestley relation p: > Vix n, We first show how to express p in terms of h. 
Two elements ĉ € X,X € Ñ” are related by p (ie. & p £) iff the inequality 
e(X) = J$ < p(ĉ) holds, equivalently, iff the left diagram below commutes laxly: 


X — vx" UX + (UX)® 
‘] N fe ve q |s. Uzi 
E L U2 4% (U2)2” 

l A T 


The duals of ĉ, ĉ; are DL morphisms x, x;: X — 2. Under duality and transposi- 
tion the left diagram corresponds to the right diagram where V is the codiagonal 
given by n-fold conjunction, i.e. it sends Q; 4 x; to \j_, vi. Writing F, = 271 (1) 
for the prime filter corresponding to a morphism z € DL(X, 2) the right dia- 
gram yields Goldblatt’s formula |16, p. 186] for the dual Priestley relation of an 
algebra h: we have ĉ p X iff h|] [; Fr,] C Fe. 

To express h in terms of p, it suffices to describe h(x) for a pure tensor 
x € (UX)®” by the universal property of the tensor product. We factor x = 
Q; ti: V71: U2 = (U2)®" — (UX)®” to see that the element h(x) corresponds 
to the following morphism representing an element of the join-semilattice UX: 


h: Q) zri: Vo: U2 = (U2)°" > (UX)®” > UX. 


Its dual is the characteristic function 
WIL, Ci 


5 y ii 
X Py Vix” k Vi (V,1)” NR, YV, Vy, 1” U, y,1” nA, Vl = 2; 


where C; = af is the clopen upset of X dual to 
a; € JSL(U2,UX) % DL(FU2, X) & Priest(X,V,1) © Priest (X, 2). 
This shows that h(x) € X = Cl, X corresponds to the clopen upset 
h(x) = {a € X | A(b1,...,bn) € pla): Vi: bi € Ci = at} € Ch(X), 
which is Goldblatt’s formula [16, p. 184] for the dual algebra of a relation p. 
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4.2 Partial Functions and Total Relations 


As a further application of the adjoint framework we characterize those operators 
whose dual Priestley relation is a partial function or a total relation, respectively. 
We achieve this by considering two splittings of the adjunction F: JSL 4 DL :U 
(Proposition 3.6 and Remark 3.7). The tensor on all categories considered is the 
tensor product of their underlying join-semilattices. 

First, split the adjunction into Q: DLo 4 DL :P and Q’: JSL 4 DLo : P’, 
where DI is the category of distributive lattices that are only bounded from 
below, and P,P’ are forgetful functors. The left adjoint Q adds a fresh top 
element to a lattice in DLo. The dual submonad QP => YV, on Priest is given by 


QPD = QPD = DL(QPD, 2) ~ DLo(PD, P2). 


Every f € DLo(PD, P2) either satisfies f(1) = 1, in which case f € D is prime, 
or f(1) = 0 but then f is the constant zero map 0!: PD — P2; note that 0! is 
clearly the bottom element in the pointwise ordering of DLo(PD, P2), so the 
monad QP just freely adds a bottom element. In particular, the dual category of 
DLo is readily seen to be equivalent to Priesto, the category of Priestley spaces 
with a bottom element, and bottom-preserving continuous monotone maps. A 
continuous relation p: X > QPX is thus simply a partial continuous function. 

Another splitting of the adjunction F 4 U is given by L: JSL, 4 DL: R and 
L’: JSL 4 JSL, : R’, where JSL; is the category of join-semilattices with both 
a bottom and top element (which are preserved by homomorphisms). The right 
adjoints R, R’ are forgetful functors. The left adjoint L maps J € JSL; to the 
distributive lattice Ug, of non-empty finitely generated upsets of J, ordered by 


reverse inclusion. The submonad LR —> YV, thus maps a Priestley space D to 
LRD = DL(LRD, 2) & ISL (RD, R2) = V} D, 


where vi is the submonad of V, taking non-empty closed downsets. Morphisms 


of type X > vY therefore are total Priestley relations. Proposition 3.6 thus 
yields the following result (the unary case is folklore, see e.g. [18, Lemma 4.6]): 


Corollary 4.5. The dual Priestley relation of a U-operator (operator morphism, 
respectively) is a partial function iff the operator (operator morphism, respectively) 
preserves non-empty meets, and total iff it preserves T. 


5 Residuation Algebras 


The abstract extended duality will now guide us in deriving a categorical duality 
between profinite ordered monoids and a full subcategory of residuation algebras 
which we call derivation algebras. This result is a non-trivial restriction of Gehrke’s 
duality [13,14] between Priestley-topological algebras and residuation algebras. 
Our result is obtained by combining two ingredients: our framework for extended 
Stone duality from the previous sections and an isomorphism between residuation 
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algebras and certain lattice coalgebras. The latter is first established for finite 
algebras via an operator on complete lattices we call tensor implication; extending 
it to locally finite algebras (Definition 5.22) then yields the desired duality with 
the category of profinite ordered monoids. To this end we introduce the notion of 
residuation morphism (Definition 5.8). The abstract extended duality then allows 
us to extend our results to relational morphisms of profinite ordered monoids 
and residuation algebras. 


5.1 The Tensor Product of Distributive Lattices Revisited 


Notation 5.1. By a lattice we always mean a bounded and distributive lattice, 
i.e. an object of DL. We often write de for d^e. The dual lattice of D is denoted 
by D°. The category of meet-semilattices (with a top element) is denoted MSL. 
Analogous to JSL it has a tensor product M X M’ and is dual to the category 
of Stone meet-semilattices [19]. From now on we denote the forgetful functors 
from DL to JSL and MSL by Uy and U,, respectively. Sometimes we omit the 
forgetful functors U, and Uy for notational brevity and just write the respective 
tensor products of the underlying semilattices as D & D’ and D X D’. 


Remark 5.2. The monad induced by the dual of Fa 4 U, sends a Priestley space 
X to its hyperspace V+X of closed upsets [1]. The comonads of the adjunctions 
Fy 1U, and Fy 4 Uy are not isomorphic but conjugate: FaU, = (FVUy(-)°)?°. 
Their restrictions to the category of boolean algebras are isomorphic since their 
dual monads satisfy V, = V = V+ — trivially so, as the order on their dual 
Priestley space is discrete. On the category of finite Priestley spaces, which are 
simply posets, V, restricts to the downset monad, which further restricts to the 
finite powerset monad on the category of finite sets (i.e., discrete finite posets). 


Remark 5.3 (Adjunctions on Lattices). By the adjoint functor theorem [25, 
Thm. V.6.1] a monotone function f: D — D’ between complete lattices pre- 
serves all joins iff it has a right adjoint f,: D’ — D, which is then given 
by f(d) = V f(d)<a’ d; dually, it preserves all meets iff it has a left adjoint 
f*: D! > D, given by f*(d') = Necta d. Finite lattices are complete, so every 
lattice homomorphism f between finite lattices has a left and a right adjoint. 
The join-irreducibles J D of a finite lattice D are precisely those elements p € D 
whose characteristic function Xp: D —> 2 (mapping x € D to 1 iff p < x) is a 
lattice morphism. The left adjoint of xp, also denoted p: 2 + D, maps 1 > p. 


Lemma 5.4. (1) The join- and meet-semilattice tensor products of distributive 
lattices D, E are isomorphic, that is, there is an isomorphism w: DE E = DRE. 


(2) Adjunctions on lattices “compose horizontally”: Given adjunctions f: DA 
E :g and f': D! 4 E' :g' on lattices, the following composites are adjoints: 
gX Ý 1 1 1 
ERE T DAD EREL DAD ERES DRD ERE Ë DED 
a fot oP RO tot Ett Lt 
EQE+,DeD FRE + DED ERE L DOD ESE T DAD 
fof fB$ ff for’ 


Monoidal Extended Stone Duality 155 


Construction 5.5. For every finite lattice D the map z @ (—): D> D8 D 
preserves all joins, so it admits a right adjoint x — (—): U,(D@D) > U, D which 
we call tensor implication. By Remark 5.3, it is given by x — T = V e@y<T y. 
Analogously, we let (—)o— x denote the right adjoint of (—) ® zx. E 


Definition 5.6. A (boolean) residuation algebra consists of a (boolean) lattice 
R € DL equipped with MSL-morphisms \: R°XR —> R and /: RXR? > R, the 
left and right residual, satisfying the residuation property: b < a\c => a< c/b. 
We call R associative if it satisfies x \ (z / y) = (x \ z2) / y for all x,y,z E€ R. A 
join-irreducible element e € JR is a unit if it satisfies e \ z = z = z / e. 


Residuals may be thought of algebraic generalizations of language derivatives, 
but as the following examples indicate they are not limited to this interpretation. 


Examples 5.7. (1) Every distributive Heyting algebra is an associative residu- 
ation algebra with residuals a \ c = a > c and c / b = b > c. 


(2) Every boolean algebra B is a non-associative residuation algebra with x\1 = 1 
and x \ z = ^z for z £1. If |B| > 1 it does not have a unit. 


(3) The dual boolean algebra X of a continuous algebra -: X x X > X ona 
Stone space X forms a residuation algebra: given clopens A,C C X, put 


A\C={xE xX |V(aeE A): a-rECh, 
C/A={x@EX|V(bE B):2-bEC}. 


(4) The regular languages Reg X over a finite alphabet X form an associative 
boolean residuation algebra with residuals given by (extended) left and right 
derivatives: K\ L = {v € S* | Kv C L} and L / K = {ve X* | vK C L}. The 
singleton empty word {e} is a unit. This example is a special case of item (3) 
obtained by taking as Stone algebra the free profinite monoid E>, 


We now introduce the notion of a residuation morphism between residuation 
algebras and also its relational generalization. 


Definition 5.8. (1) A lattice morphism f: R > S between unital residuation 
algebras is a (pure) residuation morphism if it satisfies the conditions 


f(a\ 2) < Fle)\ (2) (Forth) 
V(y, 2) € S x R: Aty2€ R): y < f(tyz) A y\ f(z) = fly \2) (Back) 
Vare<aee’ < f(z) (Unit) 


The morphism f is open if, additionally, it has a left adjoint. The category of 
unital residuation algebras with residuation morphisms is denoted Res. 


(2) A (lax) relational residuation morphism from a unital residuation algebra R 
to a unital residuation algebra S' is a morphism p € JSLi(R, S) satisfying 


p(x\ z) < p(w) \ p(z) and e< ple). 
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Unital residuation algebras with relational residuation morphisms form a category 
RelRes. 

We use the convention that for a subcategory C of Res or RelRes we denote 
the full subcategory of C with boolean carriers by BC. 


Remark 5.9. Let us provide some intuition behind the choices made in Defini- 
tion 5.8. Recall that a relational monoid morphism from a finite monoid M to N 
is a total relation p: M — PN such that p(x)p(y) C p(xy) and In € p(1m). 


(1) The notion of residuation morphism is derived from a result by Gehrke [14, 
Theorem 3.19], where it is shown to capture precisely the conditions satisfied by 
the duals of morphisms of binary Stone algebras. 


(2) We speak about relational morphisms of residuation algebras since for finite 
algebras these will dualize precisely to relational morphisms of finite monoids, 
which model inverses of surjective monoids homomorphisms [29, p. 38]: on finite 
monoids the inverse relation e~': N — PM of a surjective homomorphism 
e: M + N is the right adjoint e 4 e~' in the order-enriched category Rel with 
sets as objects and relations as morphisms, i.e. as relations they satisfy id < e~!-e 
and e- e7! < 1. Under duality the composition is reversed, so e71 dualizes to 
a left adjoint e7! + ê. As left adjoints between finite lattices are precisely the 
join-preserving functions this suggests the choice that relational morphisms of 
residuation algebras preserve finite joins (and not necessarily meets). Surjectivity 
of e is equivalent to totality of e71, which by Corollary 4.5 is equivalent to e~l 
preserving the top element. 


(3) This is also the reasoning behind the naming for open residuation morphisms: 
if e: M —> N is a continuous surjection between profinite monoids (that is, 
topological monoids in Stone), then e~': N + VM is continuous precisely iff e 
is an open map. 


For open residuation morphisms the conditions (Back) and (Forth) can be 
combined into a much simpler condition. Over finite residuation algebras this is 
particularly convenient since every residuation morphism is open. 


Lemma 5.10. Let R,S be residuation algebras. A lattice morphism f: R —> S 
is an open residuation morphism iff f*(e’) =e and it satisfies the condition 


y \ f(z) = FF) \ 2). (Open) 


Example 5.11. Let X, A be finite alphabets. Every substitution fo: X > A* 
can be extended to a monoid homomorphism f: X* — A*, and for regular 
languages L € Reg X and K € Reg A both f{L] and f~![K] are also regular. 
Then f~': Reg A — Reg is an open residuation morphism. Indeed, its left 
adjoint is f[—], and we have fHe} = {f(e)} = {e} and 


K\ f[L] = {w | Kw € fL = {w | FIK] f(w) E L} = f° (FIK]\ D). 
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5.2 Finite Residuation Algebras 


We will start by characterizing finite residuation algebras, and then generalize 
the results to locally finite residuation algebras This approach allows us to first 
introduce the key concepts and constructions of the duality on a finite level, 
and then extend them to more general structures by forming appropriate free 
completions. All results of this section apply more generally to structures with a 
complete and completely distributive lattice as carrier. 


Construction 5.12. In a finite residuation algebra R the partially applied 
residuals (x \ —),(— / y) have respective left adjoints u(a,—) 4 (x \ —) and 
u(—,y) 4 (— / y) that can be combined, by the universal property of ®, into a Uy- 
algebra u: Uy RBU R —> Uv R called multiplication. Every algebra Uy D@Uy D > 
UD on a finite lattice D has a right adjoint y: U,D —> U,(D® D) that can, 
by using the isomorphism w from Lemma 5.4, be extended to a U,-coalgebra 


Since y and ¥ are essentially the same function (differing only by the isomor- 
phism w) we refer to both as comultiplication or coalgebra structure. Conversely, 
we obtain a Uy-algebra from a comultiplication y: Ua D —> U, (D ® D) by taking 
its left adjoint. In summary, each of /, \, p, y determine each other uniquely: 


r<z/y Ss y<r\z 4> ulrt@y)<z 4 tr8y < 7), 


Lemma 5.13. In a finite residuation algebra R the residuals can be expressed 
via comultiplication y and tensor implication as x \ z = x — y(z) and z / y = 
y(z)o>— y. Conversely, the comultiplication can be expressed via residuals as 


12) = Vogt @(\D=V gg P 2 P\ 2) 


First we investigate when the comultiplication is a pure, i.e. lifts to a lattice 
morphism R > R+ R. 


Lemma 5.14. For a finite residuation algebra R, the following are equivalent: 
(1) The comultiplication is pure, i.e., y(0) = 0 and y(x V y) = y(x) V yy). 
(2) For allpe JR we have p\0=0=0 /p, and the following equations hold: 


p\(zVy)=p\zVp\y and (xVy)/p=z/pVYy/p 


(3) For all x,y E€ R: p(x @y) =0 x=0Vy=0, and uI (R+ R)| CTR. 


Next we inspect how structural identities like (co-)associativity or unitality 
translate to the other operations. Note that while the statements are to be 
expected, the proof is non-trivial due to the complication introduced by the 
seemingly innocent isomorphism w: R® R= RK R. Recall that a coalgebra 
c: URR > U RX U,R is coassociative if (c X id) - c = (id X c) - c and counital if 
it is equipped with a counit € € DL(R, 2) such that (e X id) -c = id = (id X e) - c. 


158 F. Birkmann, S. Milius, H. Urbat 

Lemma 5.15. The following are equivalent for a finite residuation algebra R: 
(1) The comultiplication on R is coassociative and has a counit. 

(2) The residuals are associative and R has a unit. 


(3) The multiplication u is associative and has a unit, i.e. a join-irreducible 
e € JR satisfying u(e ® —) = id = pp(— @e). 


These lemmas suggest the following definitions. 


Definition 5.16. (1) A finite residuation algebra R is pure if it satisfies (one 
of) the equivalent conditions of Lemma 5.14. 

(2) A finite residuation algebra R is a finite derivation algebra if it is pure, 
associative and has a unit. The respective full subcategories of Res; and RelRes; 
are denoted by Der; and RelDers. 

(3) A (not necessarily finite) U,-coalgebra 4: UAC > UnC X UAC is a Up,- 


comonoid if its coassociative and counital, and a (lattice) comonoid if ¥ is pure. 


In order to extend the correspondence of (finite) residuation algebras and 
U,-coalgebras to a categorical equivalence we introduce appropriate morphisms. 


Definition 5.17. (1) A pure morphism from a counital U,-coalgebra (C, 7, €) 
to (C’,4’,¢€) is a lattice morphism f: C —> D satisfying (f X f)-7=%- f and 
c=. f. 


UC Uai gira UC 2 U,C 
I N le 
Unf QUnf ; ; 
ULC HU, C — U,C K U,C U, 2 


The category of counital U,-coalgebras with pure morphisms is denoted by 
Coalg(U,) and its full subcatgegory of U,-comonoids by Comon(U, ), again 
with the full subcategory Comon of comonoids. 


(2) Let C and C” be comonoids. A (lax) relational morphism from C to C” is a 
morphism p € JSL; (C, C”) satisfying (p@ p)-y<7'-p and €< €- p, i.e. the 
following diagrams in JSL commute laxly: 


Ue 2 UC — U0" 


oR N. aa 
Uy2 


UC 8 UC h WC’ @UvCc’ 
Comonoids with relational morphisms form a category RelComon. 
Theorem 5.18. The following categories are isomorphic: 


Coalg,(U,) S Resp, Comon; = Derg and RelComon; = RelDer;. 
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Proof (Sketch). On objects the isomorphism swaps between residuals and comul- 
tiplication; the residual unit is left adjoint of the counit. The first isomorphism 
restricts to the second by Lemmas 5.14 and 5.15. On morphisms one proves that 
a lattice morphism f: C — C’ is a pure coalgebra morphism iff it is an (open) 
residuation morphism, and if C and C” are comonoids, then p € JSL, (C, C”) is 
a relational comonoid morphism iff it is a relational residuation morphism. 


From Theorem 5.18 we obtain the following dual characterization of finite 
ordered monoids; it restricts to the order-discrete setting of ordinary finite 
monoids and finite boolean derivation algebras. 


Theorem 5.19. (1) The category of finite ordered monoids is dually equivalent 
to the category of finite derivation algebras (or finite lattice comonotds): 


OrdMon,; ~°? Comon,; = Dere. 


(2) The category of finite ordered monoids with relational morphisms is dually 
equivalent to the category of finite derivation algebras (or finite lattice comonoids) 
with relational morphisms. 


RelOrdMon; ~°? RelComon; & RelDers. 


Proof. The first statement is a trivial extension of Theorem 5.18 by (finite) 
Priestley duality since finite ordered monoids dualize to finite lattice comonoids. 
For item (2) note that a relational ordered monoid morphism (M,-m, 1m) > 
(N,-nv,1y) is a total relation p: M —> DN (where D is the downset monad) 
making the following diagrams commute laxly: 


MxM e: >s M 1 —“> M 
J L [> 2» L P 
DN x DN Š> D(N x N) 23 DN N — DN 


If we view N as a finite Priestley space, then DN = V, N, so the dual of p under 
(order-enriched) extended duality is a relational morphism p~ € JSL; (N, M) of 
finite lattice comonoids, or equivalently, a relational residuation morphism. 


5.3 Locally Finite Residuation Algebras 


The main complication in the generalization from finite to infinite structures 
comes from the reliance on adjoints, as these may not exist anymore on infinite 
lattices. The prime example of a residuation algebra in automata theory suggests 
a local translation between residuals and comultiplication: 


Example 5.20. It is well-known that the boolean algebra Reg X of regular 
languages dualizes under Stone duality to the free profinite monoid X* (see 
Pippenger [27]). The multiplication u: X* x X* — X* of profinite words dualizes 
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under Stone duality to a comultiplication ~t: Reg © —> Reg X + Reg X on 
regular languages defined on L € Reg X by 


BL) = V pesya, E18 EI L. (5-1) 


Here Syn, is the syntactic monoid of L, whose elements are the equivalence 
classes of the equivalence relation on X* defined by v =p w iff v,w belong to the 
same residuals K \ L / M. Gehrke [13, Thm. 15] has shown that, under Stone 
duality, Syn; dualizes to the residuation ideal generated by L € Reg X. 


Definition 5.21. A residuation ideal of a residuation algebra R is a sublattice 
I > R such that for all z € J and z € R one has z \ z,z / x € R. We denote the 
residuation ideal generated by a subset X C R by \X/. 


Residuation ideals were used by Gehrke [14] to characterize quotients of Priest- 
ley topological algebras. Note that in the formula (5.1) for the comultiplication 
on regular languages it is crucial that the residuation ideal \{L}/ generated by a 
single regular language L is finite, as otherwise the join might not exist. This 
leads to the following restriction. 


Definition 5.22. (1) A residuation algebra R is locally finite if every finite 
subset of R is contained in a finite residuation ideal of R. 


(2) A U,-coalgebra C is locally finite if every finite subset of C is contained in 
a finite subcoalgebra of C. The category of locally finite comonoids is denoted 
Comont. 


Note that not every residuation algebra is locally finite, consider for example 
an infinite boolean algebra in Example 5.7(2). 


Proposition 5.23. (1) Every locally finite residuation algebra R yields a locally 
finite Un -coalgebra 4 : Ua R > Un (R 8 R) with comultiplication given by 


Ke) = (ta 8al) =V ala) ealas V 


A(p) 2 ta(p \ z) 


L L 
xrEA pETA 


for any finite residuation ideal ta: A > R containing z (here ya is the comulti- 
plication on A as in Construction 5.12). 


(2) Every locally finite U,-coalgebra (C,y) yields any locally finite residuation 
algebra with the left residual given by x \y z = 14 (x \a 2) = ta (x£ — ¥(z)) for 
any finite subcoalgebra ta: A > C containing x,z (here \4 is the residual on 
A as given by Construction 5.12). The residual has a canonical presentation as 
x \y Z = bz (te (a) \ z), where uz: (z) => C is the smallest (finite) subcoalgebra 
containing z. The right residual is defined analogously. 


(3) These translations are mutually inverse. 


Proposition 5.23 shows that every locally finite residuation algebra carries 
a unique U,-coalgebra structure and vice versa. We may thus translate at will 
between the residuals and comultiplication as in the finite case and omit the 
subscripts. We extend Lemmas 5.14 and 5.15 to locally finite structures: 
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Lemma 5.24. Let R be a locally finite residuation algebra. 
(1) Finite residuation ideals correspond to finite subcoalgebras. 
(2) The residuals are associative iff the comultiplication is coassociative. 
(3) The residuals have a unit iff the comultiplication is counital. 
(4) The comultiplication is pure iff every finite residuation ideal is pure (see 


Definition 5.16). 


Remark 5.25. Lemma 5.24(4) characterizes locally finite residuation algebras 
with a pure comultiplication. By extended duality, their dual Priestley relation 
is functional. We note that Gehrke [14, Proposition 3.15] presented a necessary 
and sufficient condition for a general residuation algebra R to have a functional 
dual relation, namely join-preservation at primes: 


VF € DL(R, 2): V(a € F),V(b,c € R): da’ € F: a\ (bV c) < (a \b) V (a \ Cc). 


One can show that every locally finite residuation algebra satisfying Lemma 5.24(4) 
is join-preserving at primes. 


Definition 5.26. A residuation algebra R is a derivation algebra if it is locally 
finite, associative, unital and every finite residuation ideal J is pure. The ensuing 
full subcategories of Res and RelRes are denoted Der and RelDer. 


Theorem 5.27. (1) The category of locally finite residuation algebras and resi- 
duation morphisms is isomorphic to the category of locally finite unital U,- 
coalgebras and pure coalgebra morphisms. 


(2) The isomorphism restricts to the full subcategories of derivation algebras and 
locally finite comonoids. 


(3) The categories of derivation algebras and relational residuation morphisms 
and locally finite comonoids with relational morphisms are isomorphic. 


Combining this characterization with our approach to extended Priestley 
duality we establish a duality between profinite ordered monoids and derivation 
algebras, and extend it to relational morphisms. Conceptually, this general duality 
is an extension of the finite duality OrdMon; ~°? Comon; = Der, by forming 
suitable completions: Profinite ordered monoids are the Pro-completion (the free 
completion under cofiltered limits) of the category of finite ordered monoids; 
dually a routine verification establishes that lattice comonoids (and therefore also 
derivation algebras by Theorem 5.27(2)) form Ind-completions (free completions 
under directed colimits) of their respective subcategories of finite objects. 


Proposition 5.28. The category of locally finite comonoids forms the Ind- 
completion of the category of finite comonoids: 


Comon; ~ Ind(Comon,). 


We define a Priestley relational morphism between profinite ordered monoids 
X,Y to be a Priestley relation p: X — VY such that p(x)o(x') C p(xx') and 
In € plm). 
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Theorem 5.29. (1) The category of derivation algebras is dually equivalent to 
the category of profinite ordered monoids: 


Der = Comon; ~°? ProfOrdMon. 


(2) The category of derivation algebras and relational residuation morphisms 
is dually equivalent to the category of profinite ordered monoids and Priestley 
relational morphisms: 


RelDer ~ RelComon; ~°? RelProfOrdMon. 


Remark 5.30. (1) Theorem 5.29 clearly restricts to profinite monoids with 
Stone relational morphisms and boolean derivation algebras. It is well-known 
that every Stone monoid is profinite (see e.g. [20]). So dually, every boolean 
comonoid is locally finite. 
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(2) All results of Section 5 hold analogously for the extension of the “discrete 
duality between posets (or sets) and algebraic completely distributive lattices (or 
completely atomic boolean algebras) along the free-forgetful adjunction between 
completely distributive lattices and complete join-semilattices. This yields a 
duality between the category of all (ordered) monoids and (completely distributive 
lattices) completely atomic boolean residuation algebras with open residuation 
morphisms. Moreover, this duality also can be extended to relational morphisms. 


6 Conclusion and Future Work 


We have presented an abstract approach to extending Stone-type dualities based 
on adjunctions between monoidal categories and instantiated it to recover and 
generalize extended Priestley duality. Guided by these foundations we have 
investigated residuation and derivation algebras and proved a duality between 
the latter and (ordered) profinite monoids, Moreover, we have extended this 
duality to relational morphisms. 

Relational morphisms are an important tool in algebraic language theory, 
notably for charaterizing language operations algebraically. For instance, aperiodic 
relational morphisms are tightly connected to the concatenation product and the 
star operation on regular languages. In future work we intend to apply the new 
duality-theoretic results on relational morphisms to illuminate such connections, 
much in the spirit of the duality-theoretic persepective of Eilenberg’s Variety 
Theorem by Gehrke et. al. [15]. 

Another goal is to apply our abstract duality framework beyond classical 
Stone and Priestley dualities. Specifically, we aim to develop an extended duality 
theory for the recently developed nominal Stone duality [4], which would allow 
to generalize our present results on residuation algebras to the nominal setting 
and uncover new results about data languages. 

A conceptually rather different dual characterization of the category of profi- 
nite monoids and continuous monoid morphisms in terms of semi-Galois categories 
has been provided by Uramoto [35]. Extending this result to relational morphisms, 
similar to our Theorem 5.29, is another interesting point for future work. 
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Abstract. We introduce a compositional framework for convex analysis 
based on the notion of convex bifunction of Rockafellar. This framework 
is well-suited to graphical reasoning, and exhibits rich dualities such 
as the Legendre-Fenchel transform, while generalizing formalisms like 
graphical linear algebra, convex relations and convex programming. We 
connect our framework to probability theory by interpreting the Laplace 
approximation in its context: The exactness of this approximation on 
normal distributions means that logdensity is a functor from Gaussian 
probability (densities and integration) to concave bifunctions and maxi- 
mization. 


Keywords: convex analysis - category theory - categorical probability 


1 Introduction 


Convex analysis is a classical area of mathematics with innumerous applications 
in engineering, economics, physics, statistics and information theory. The central 
notion is that of a convex function f : R” > R, satisfying the inequality f (tx + 
(1—t)y) < tf(x) + (1 — t)f(y) for all t € [0,1]. Convexity is a useful property 
for optimization problems: Every local minimum of f is automatically a global 
minimum. Convex functions furthermore admit a beautiful duality theory; the 
ubiquitous Legendre-Fenchel transform (or convex conjugation) defined as 


Pe) Saupe Fe) 


encodes f in terms of all affine functions (x*, £) — c majorized by f (here (—, —) 
denotes the standard inner product on R”). The function f* is convex regardless 
of f, and under a closedness assumption we recover f** = f. 

While convex analysis is a rich field, its compositional structure is not readily 
apparent; the central notion, convex functions, is not closed under composition. 
The notion which does compose is less well known: a convex bifunction, due 
to [27], is a jointly convex function F : R™ x R” — R of two variables. Such 
bifunctions compose via infimization 


(F o G)(x, 2) = inf {F(y, z) + G(s, y)} 
© The Author(s) 2024 
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Categorical Methods In this work, we will study bifunctions and their associated 
dualities in the framework of category theory. Graphical methods are ubiqui- 
tous in engineering and statistics, and can used to derive efficient algorithms 
by making use of the factorized structure of a problem. The language of props 
and string diagrams unifies these methods, as a large body of work on graphical 
linear algebra and applied category theory shows [2,1,19,7]. We extend these 
methods to problems of convex analysis and optimization. Our category of bi- 
functions subsumes an array of mathematical structures, such as linear maps and 
relations, convex relations, and (surprisingly) multivariate Gaussian probability. 


Fig. 1. Addition of independent normal variables X,Y. Left: pdf and convolution, 
right: logpdf and sup-convolution 


Applications to Probability Theory Convex analysis offers a rich perspective on 
Gaussian (multivariate normal) probability distributions: The log-density h(x) = 
log f(x) of a Gaussian random variable is a concave function of the form® 
(z — u)? 
h(x) = — 
(x) 552 

It turns out that anything we can do with Gaussian densities and integration can 
instead be done with logdensities and maximization. For example, to compute 
the density of a sum of independent variables, we may take a convolution of 
densities, or instead compute a sup-convolution of logdensities (see Fig. 1), as 


jog J fix 2) fy (2 — 2)dz = sup {ħx(2) + hy (2 — 2)} 


This is highly particular to Gaussians. We can elegantly formalize this state- 
ment in categorical terms, as our main theorem states: Logprobability defines a 
functor from Gaussian probability to concave bifunctions (Theorem 5) 

In this sense, the essence of Gaussians is captured by concave quadratic 
functions. By extending our viewpoint to partial concave quadratic functions, we 


3 we intentionally disregard a scalar +C 
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obtain a generalized notion of Gaussian relation which includes improper priors. 
Such entities are subtle to describe measure-theoretically, but straightforward 
in the convex analytic view. The duality theory of bifunctions generalizes the 
duality of precision and covariance, and more generally connects to the notion 
of cumulant-generating function in probability theory. 

We elegantly formalize the connections between convex analysis and proba- 
bility theory using the language of Markov categories [17], which are a categorical 
formalism for probability theory, and have close connections to the semantics of 
probabilistic programs [30]. 


Contribution and Outline This paper is intended to serve as a high-level roadmap 
to a categorical treatment of convex analysis. Our aim is to spell out the un- 
derlying structures, and present a diverse range of connections, especially with 
diagrammatic methods and categorical probability. For the sake of presentation, 
we choose to stick to general statements and keep some technical notions (such 
as regularity conditions) informal. Spelling out the details in a concrete setting 
is a starting point for future developments. We elaborate one such particular 
setting in detail, namely Gaussian probability. 


We begin §2 by recalling the relevant notions of convex analysis, and proceed 
to define and study the categorical structure of bifunctions in §3. This includes 
two structures as a hypergraph category and the duality theory of §3.1. 

In §4, we elaborate different examples of categories which embed in bifunc- 
tions, such as linear and affine algebra, linear algebra, convex relations and con- 
vex optimization problems. In each case, the embedding preserves the relevant 
categorical structures and dualities. In particular, we show that the theory of 
bifunctions is a conservative extension of graphical linear algebra [25]. 

In §5 we begin making connections to probability theory. We recall Gaussian 
probability from a categorical point of view, and construct the embedding functor 
to bifunctions. We discuss how partial quadratic functions can be seen as an 
extension of Gaussian probability beyond measure theory. 

We conclude with §6-7 discussing the wider context of this work, elaborating 
connections of probability and convex analysis such as the Laplace approxima- 
tion and cumulant generating functions, and the idea of idempotent analysis as 
a ‘tropical limit’ of ordinary analysis. 


2 Overview of Convex Analysis 


The following section is a brief overview of standard material in convex analysis; 
all propositions and conventions are taken from [27]. 

Caveat: An important feature of convex analysis is that it deals with formal 
infinities +00, —oo in a consistent fashion. This is crucial because optimization 
problems may be unbounded. Traditionally, one considers the extended real num- 
bers R = [—00, +00] and extends the usual laws of arithmetic to them. The case 
(+00) + (—oo) is left undefined and carefully avoided like 0/0 in real analysis. 
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A more systematic approach [37,18] is based on enriched category theory, and 
endows R with the structure of a commutative quantale, which gives it totally 
defined operations with a particular arithmetic. 

A more serious caveat is that many results in convex analysis require specific 
regularity assumptions to hold. As these assumptions are not the focus of the 
present paper, so we will state some big picture theorems in §3 under reservation 
of these conditions. We then elaborate an array of concrete examples §4-5 where 
we make sure that all regularity conditions are indeed satisfied. We discuss this 
drawback in §7. 


A subset A C R” is convex if for all x,y € A and t € [0,1], we have ta + 
(1—t)y € A. The epigraph of a function f : R” — R is the set epi(f) = 
{(z,y) E€ R"t! : y > f(ax)}. We say that f is convex if epi(f) is a convex subset 
of R"+!. This is equivalent to the well-known definition from the introduction, 
while accounting for infinities. We say that f is concave if (—f) is convex. 


Example 1. The following functions are convex: linear functions, |x|, 27, exp(z), 
— ln(x). For a convex subset A C R”, the convex indicator function ĝa : R” > R 


is defined by 
0 TEA 
ô ZF 
a(z) a rgA 


We also write indicator functions using modified Iverson brackets as {|x € A} = 
ôa (x). The concave indicator function of A is —ô4 (x). 

The infimal convolution of convex functions f,g : R” — R is defined by 
(f O g)(x) = inf, {f(x — y) + g(y)}. The convex function f is called closed if 
epi( f) is a closed subset of R”+!; this is equivalent to f being lower semicontin- 
uous. 


2.1 Conjugacy — the Legendre-Fenchel transform 


Definition 1. For a convex function f : R” — R, its conver conjugate f* : 
R” > R is the convex function 


P(e) = sup {ieee} — f(a) 


For a concave function g : R” > R, its concave conjugate g* : R” — R is the 
concave function 


g” (x*) = inf {(a", x) — g(x)} 
Note that if g = —f then g*(a*) = —f*(—a*) 


Geometrically, f* encodes information about which affine functions (a*,—) — c 
are majorized by f. It is thus natural to view f* as a function on covectors 
x* € (R")*. This is for example done in [37], but in order to keep notation 
consistent with [27], we make the traditional identification (R”)* = R” via the 
inner product, and the notation x* is purely decoration. The Legendre-Fenchel 
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transform has applications in many areas of mathematics and physics [34], such 
as the Hamiltonian formalism in mechanics, statistical mechanics or large devi- 
ation theory (e.g. §6.2). 


A closed convex function f is the pointwise supremum of all affine functions 
h < f [27, 12.1]. This allows them to be recovered by their Legendre transform 


Proposition 1 ([27, Theorem 12.2]). For any convex function f : R” > R, 
f* is a closed convex function. We have f** = f if and only if f is closed. 


For arbitrary functions f, the operation f + f** is a closure operator which we 
denote by cl(f). This is the largest closed convex function majorized by f. 


Example 2. The absolute value function f(x) = |x| is convex and closed. The 
supremum sup, {cx — |x|} equals 0 if |c| < 1, and co otherwise. Hence f*(c) = 
{lel < 1}, and f** = f. 


Example 3. Let f(x) = ax? for a > 0. Then x > c- g — az? is differentiable and 
has a maximum at x = c/2a. We obtain f*(c) = +œ. In particular, we see that 
the function f(x) = $2? is a fixed point of the Legendre transform. 


Proposition 2 ([27, Theorem 16.4]). If f,g are closed convex functions, then 


* 


under certain regularity conditions (f O g)* = f* + g* and (f +g) =f* Og". 


3 Categories of Convex Bifunctions 


We now come to the central definition of this article, namely that of convex (or 
concave) bifunctions. This concept is due to [27] and scattered throughout his 
book. 

A bifunction F from R™ to R” is the convex analysis terminology for a 
curried function R™ —> (R” — R). The uncurried function F : R™+" — R is 
referred to as the graph function of F. We will suppress the partial application 
and write F(x)(y) and F(z, y) interchangeably. 


Definition 2. A bifunction F from R™ to R” is called convex (or concave, 
closed) if its graph function F : R™*" — R has that property. The closure 
operation cl(F') is applied on the level of graph functions. We denote a convex 
bifunction by F : R™ — R” and a concave bifunction by F : R™ — R”. 


Bifunction composition is known as product in [27, § 38]. 


Definition 3 (Categories of bifunctions). We define a category CxBiFn of 
convex bifunctions as follows 


— objects are the spaces R” 
— morphisms are convex bifunctions R™ — R” 
— the identity R” — R” is given by the indicator function 


idn (x,y) = {£ = y} 
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— composition is infimization over the middle variable 
(F © G)(x,2) = inf {G(,y) + F(y, 2)} 


Analogously, the category CvBiFn of concave bifunctions is defined as 


— objects are the spaces R” 
morphisms are concave bifunctions R™ — R” 
— the identity R” — R” is given by the concave indicator function 


~idn(z,y) = -Åz = yh 


— composition is supremization over the middle variable 


(F © G)(z,z) = suj {G(a,y) + F(y,z)} 


Proof (of well-definedness). This construction is a subcategory of the the cate- 
gory of weighted relations Rel(Q) taking values in a commutative quantale Q 
[3, 12, 23], where Q = R are the extended reals. It suffices to verify that con- 
vex bifunctions are closed under composition, tensor (addition) and contain the 
identities ([27, p. 408}). 


We will write bifunction composition as F o G when it is clear from context 
whether we use the convex or concave variety. We will write I for the unit space 
R°, and O for its unique element. 


Example 4. The states (morphisms J — R” out of the unit) are in bijection with 
convex functions f : R” — R, as are the effects R” — I. States and effects in 
CvBiFn are in bijection with concave functions f : R” > R. 


3.1 Duality for Bifunctions 


Unless otherwise stated, theorems phrased for convex bifunctions will hold for 
concave bifunctions by selecting the appropriate versions of the operations. 
The duality theory of convex functions extends to bifunctions as follows. 


Definition 4 ([27, §30]). The adjoint of a convex bifunction F : R™ — R” is 
the concave bifunction F* : R” — R™ defined by 


FM(y", 2") = inf (F(x, y) + (2", 2) — (y", 9) 


The adjoint of a concave bifunction is convex and uses sup instead of inf. The 
adjoint of the convex bifunction F is related to the conjugate of its graph function 
F using the formula F*(y*,x*) = —F*(—2*,y*). (Note the slight asymmetry that 
one input is negated) 


The analogue of Proposition 1 for bifunctions is as follows 
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Proposition 3 ([27, Theorem 30.1]). For any convex bifunction F, the ad- 
joint F* is a closed concave bifunction, and we have F** = cl(F). In particular, 
if F is a closed convex bifunction, then F** = F. 


Theorem 1 ([27, Theorem 38.5]). Under regularity assumptions, the adjoint 
operation respects composition. That is, for F : R™ — R” and G : R” — RF, we 
have 


(Go F)*=F*oG 


That is, the adjoint operation defines a pair of mutually inverse functors 


We indicate with dashed arrows that the functoriality depends on regularity as- 
sumptions. 


For the interested reader, the regularity assumptions in Theorem 1 include 
closedness, as well as properness and certain (relative interiors of) domains of the 
involved bifunctions intersecting [27, § 38]. These assumptions are not necessary 
conditions. 

As a corollary of functoriality, we can derive the following well-known fact: 


Corollary 1 (Fenchel duality). Let f : R” — R be conver, g : R” —> R 
concave, and let f*,g* be their convex and concave conjugates respectively. Then 
under sufficient regularity assumptions, we have 


inf { f(x) — g(x)} = sup {g"(a") — f @*)} 
Proof. Consider the convex function h = —g and form the state sp : I — 
R”, sp(0, x) = f(x) and effect e, : R” — I, en(x,0) = h(x). The proof proceeds 


by using functoriality to compute the scalar (ep, © s D= (85 © ež) in two ways: 
On the one hand, we have 


(en © 87)(0,0) = inf {s/(0,2) + en(a,0)} = inf {f (x) — g(x)} 
On the other hand, we express the adjoints in terms of the conjugates f*, g* 


s$ (2*0) = inf {s(0, x) — ioe) } = =f" Ge) 
e} (0, xz*) = inf {en(x, 0) + (x*, x£) } = g* (x*) 


The adjoint acts as the identity on scalars, so we obtain 


inf { f(x) — g(x)} = (en © sp)* (0,0) = (s} © e})(0,0) = sup gle SE E 
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3.2 Hypergraph Structure and Symmetries 


Bifunctions can not only be composed in sequence, but also in parallel. The 
relevant structure is that of a symmetric monoidal category (C,@,J). In this 
work, we are dealing with a particular simple form of such categories called a 
prop. A prop C is a strict monoidal category which is generated by a single object 
R so that every object is of the form R®” for some n € N. The monoid of objects 
(ob(C), &, T) is thus isomorphic to (N, +, 0). 


Proposition 4. Convex bifunctions have the structure of a prop, generated by 
the object R 


1. The tensor is R” @ R” = Rt” 

2. The unit is I = RÌ. 

3. The tensor of bifunctions is given by addition: If F : R™! — R™,G:R™ = 
R”? then FQG:R™t™ R12 is defined as 


(F 8 G)((£1, £2), (y1, Y2)) = F(21, 41) + G(r2, y2) 


Proof (of well-definedness). General fact about categories of weighted relations 


Re1(Q) ([23]). 


Symmetric monoidal categories are widely studied and admit a convenient 
graphical language using string diagrams [28]. It is useful to consider further 
pieces of structure on such a category 


1. in a copy-delete category [11], every object carries the structure of a com- 
mutative comonoid copy y : X + X & X and discardx : X —> I. This lets 
information be used in a non-linear way (in the sense of linear logic). 

2. in a hypergraph category [14], every object carries the structure of a special 
commutative Frobenius algebra 


Every hypergraph category is in particular a copy-delete category. The pieces of 
structure of a hypergraph category are often rendered as cups and caps in string 


diagrams 


copy discard multiply unit 


subject to equations such as the Frobenius law 


a een 


This gives rise to a rich graphical calculus, which has been explored for a 
number of engineering applications like signal-flow diagrams or electrical circuits 
(25, 8, 7,9, 2, 1] 
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Proposition 5. CxBiFn has the structure of a hypergraph category in two dif- 
ferent ways, which we call the additive and co-additive structure. That is, every 
object carries two different structures as a special commutative Frobenius algebra 


1. The additive structure is given by 


unit : J — R”, 

discard : R” — T, 

copy : R” — R” @ R” 
comp : R” ® R” — R”, 


unit(0, x) = 0 

discard(x, 0) = 0 
copy(#,y,2) = {x =y = z} 
comp(2,y, z) = {x = y = 2} 


2. The co-additive structure is given by 


zero: I > R”, zero(0, x) = {|x = 0} 
cozero : R” > J, cozero(x,0) = {|x = 0} 
add: R” & R” — R”, add(z, y, z) = {xr +y = z} 


coadd : R” — R” & R”, coadd(z, x,y) = {a+y = zh} 


The analogous structures on CvBiFn use concave indicator functions instead. 


We can motivate the names of the hypergraph structures by observing how 
multiplications acts on states. This duality is clarified in what follows. 


Example 5. Let f,g : I — R” be two states. Then 
(copy © (F 8 g))(2) = inf {1f(2) + gly) + lz = y = z}} = F(z) + g(2) 
(addo (f 8 9))(z) = inf {f(@) + oy) +de +y = zh} = f(2) O of) 


Definition 5. The dagger of a bifunction F : R™ — R” is given by reversing 
its arguments 
Ft: R? SR”, FÝ (y, x) = F(x,y) 


The inverse of a bifunction F : R™ — R” is the concave bifunction [27, p. 384] 
F(x,y) = -F (y, 2) 
Both these operations define involutive’ functors 
(—)' : CxBiFn°? — CxBiFn, (—), : CxBiFn°? — CvBiFn 
The functor (—)' is a dagger functor in the sense of [29]. 


Proposition 6 ([27, p. 384]). The operations of inverse and adjoint commute, 
i.e. for F : R” — R” we have (F*), = (Fi)*. 


4 i.e. applying the appropriate version of these operations twice is the identity 
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The composite operation F defines another covariant functor CxBiFn —> 
CxBiFn, which we now interpret: As is customary in graphical linear algebra, we 
render the two hypergraph structures as follows 


copy discard unit comp 

(1) 
C ke Te 
coadd cozero Zero add 


We refer to the additive structure as ‘black’ (e) and the co-additive one as 
‘white’ (o). This presentation reveals an array of symmetries (mirror-image and 
color-swap”), which we are relating now: 


Theorem 2. The adjoint operation interchanges the additive and co-additive 
structure. That is we have functors of hypergraph categories 


(—)* : (CxBiFn°?, e) — (CvBiFn, o) 
(—)* : (CxBiFn°?, o) + (CvBiFn, è) 


Note that the opposite of a hypergraph category is again a hypergraph category 
where cups and caps are interchanged. 


Proof. Follows from the results in §4.1, as the hypergraph structures are induced 
by linear maps. 


In terms of the generators (1), the mirror image is given by the (—)' functor. 
Both hypergraph structures consist of {-Frobenius algebras, meaning that (—)t 
is a functor of hypergraph categories CxBiFn°P — CxBiFn. 

The color-swap operation is given by the inverse adjoint F*, which gives a hy- 
pergraph equivalence (CxBiFn, e) — (CxBiFn,o). This equivalence does however 
not commute with t, i.e. is not an equivalence of dagger hypergraph categories. 


4 Example Categories of Bifunctions 
We now elaborate example subcategories of bifunctions on which functoriality 


and duality work as desired (that is, all regularity conditions apply). 


5 we will discuss these symmetries in more detail in Section 4.1 
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4.1 Linear Algebra 


The identities and dualities of convex bifunctions generalize those of linear alge- 
bra. Let A: R” — R” be a linear map. The convex indicator bifunction of A is 
defined as 


Fa(x,y) = ly = Aah 
The following facts hold [27, p 310]: 


1. For composable linear maps, A, B we have F4g = F4 o Fg 
2. The adjoint F4 is the concave indicator bifunction of the transpose AT 


Fa(y*,2*) = —{\a* = ATy*h 


3. if A is invertible, then the inverse (F'4), is the concave indicator bifunction 
associated to the inverse A71. In that case, Proposition 6 generalizes the 
identity (A71)7 = (AT)7?. 


In more categorical terms, let Vect denote the prop of the vector spaces R” 
and linear maps. This is a copy-delete category equipped with the linear maps 
A: R” > R” OR” and!: R” > R°. For a linear map A: R™ > R”, define 


F4 : R” —R", Fa(a,y) = fy = Aa} 
Ga : R” —R”,Galy,2) = {|x = ATy} 


Theorem 3. We have a commutative diagram of functors between copy-delete 
categories 
Vect 


(CvBiFn®?,o) 7 Ses ” (CxBiFn, e) 
Proof. Functoriality and commutativity follow from the above facts. For the 
copy-delete structures, notice that copy, delete, add, zero are the indicator bifunc- 
tions of the linear maps A and !. The transpose of A is summation (z, y) œ> x+y. 


We call a diagram like (2) a duality situation. The dashed arrows indicate 
that, while (—)* is neither a functor nor idempotent on all bifunctions without 
further conditions, everything works out on the image of FG respectively. We 
could thus obtain a genuine commutative diagram of functors by characterizing 
these images exactly (which we refrain from doing here for the sake of simplicity). 


Linear Relations Graphical Linear Algebra [25] is the diagrammatic study of the 
prop LinRel of linear relations, which are relations R C R™ x R” that are also 
vector subspaces. This category is a hypergraph category using the two structures 
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shown in (1), and the operations mirror-image and color-swap are defined for 
linear relations via relational converse and a twisted orthogonal complement 


Rİ = {(y,2) : (@,y) € R} 
R° = {(z*,y") : V(x, y) ER, (x*, 2) = (y",y) > O} 


The operations (—)' and (—)° commute and define a composite contravariant 
involution (—)* : LinRel°P — LinRel. The following theorem shows that bifunc- 
tions are a conservative extension of graphical linear algebra. 


Theorem 4. If we embed a linear relation R C R™ x R” via its indicator func- 
tion as a bifunction Ip : R™ — R”, then we have a commutative diagram 


LinRe1°P (-)* LinRel 


In addition, the functor I preserves both hypergraph structures. 


Affine Relations Graphical linear algebra has been extended to affine relations 
[6]; those are affine subspaces R C R™ x R”. This still forms a hypergraph 
category with both structures e,o, however the color-swap symmetry of linear 
relations is broken. That is because the affine generator 1 : 0 > 1 representing 
the affine relation {(0,1)} does not have an obvious color-swapped dual; affine 
subspaces are not recovered by their orthogonal complements. 

The embedding into bifunctions suggests an avenue to recover such a sym- 
metry: Taking the embedding (3) as a starting point, the indicator bifunction 
of 1 is f : I — R, f(0,x) = {jx = 1}. Its adjoint is f*(x*,0) = —2*, which 
is a perfectly well-defined bifunction but not the indicator bifunction of any 
affine relation. This suggests that an extension of affine relations with color- 
swap symmetry can be obtained using a category of partial affine function (e.g. 
(27, p. 107]) but details are to left for future work. We will discuss the case of 
partial quadratic functions in §5.2. 


4.2 Convex Relations 


Generalizing the previous example even further, a convex relation R C R™ x R” 
is a relation which is also a convex subset of R™*”. Convex relations are closed 
under the usual relation composition and thus form a prop CxRel [3, 12, 23). 
Every linear relation is in particular convex, and like linear relations, convex 
relations embed into convex bifunctions via the indicator function. 
We sketch a certain converse to this embedding: The space (R,+,0) is a 
monoid object in CxRel. We consider the ‘writer’ monad T : CxRel — CxRel 
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associated to that monoid, ie. T(R™) = R®+1, If S C R” x R"*! and R C 
R” x R*+! are Kleisli arrows, then Kleisli composition takes the following form 


ReS= {(x, 2, tı + tə) : (x, y, t1) € S, (y, z, ta) € R} 


Given a convex bifunction F : R™ — R”, the epigraph of its graph function 
epi(F) C R™ x R”*! is thus a Kleisli arrow for T. Under sufficient regularity 
assumptions, this is functorial, and we have an embedding epi : CxBiFn — 
CxRelr. 


4.3 Ordinary Convex Programs 


We briefly discuss the historical origins of bifunctions in convex optimization 
(27, § 29-30]: For simplicity, we say that a ordinary convex program P is a 
minimization problem of the form 


inf{ f(a): a2 €R”,gi(x) < 0,..., 94 (x) < 0} 


where the objective function f and the constraints g1,..., gz : R” > R are finite 
convex functions. The bifunction associated to P is defined as 


k 
Fp: R* —> R”, Fp(v, x) a f(z) +) Alfa) < vil} 


The inputs of v € R* can be thought of as perturbations of the constraints. The 
so-called perturbation function of P is the parameterized minimization prob- 
lem (inf Fp)(v) = inf, {Fp(v,x£)}. The convex function Fp(0,—) represents the 
unperturbed problem and (inf Fp)(0) is the desired solution. Note that in cate- 
gorical language, the perturbation function is straightforwardly obtained as the 
bifunction composite (discard o Fp) : R? — J, or graphically 


RE Fp o 


The associated bifunction Fp contains all information about the problem P, and 
allows one to find the dual problem P* by taking its adjoint. This way one can 
think of any bifunction R* — R” as a generalized convex program ([27, p. 294]). 


Example 6 ([27, p. 312]). Consider a linear minimization problem P of the form 
inf{(c, x) : b— Ax < 0} 
The associated bifunction and its adjoint are 
F(v, x) = (c, x£) + {x > 0,b— Az < vf} 
F*(x*,v*) = (b, v*) — u* > 0,c— ATv* > a*f 
which is the concave bifunction associated to the dual maximization problem 


sup{ (b, y) : y > 0,c— ATy > 0} 
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5 Gaussian Probability and Convexity 


We now study the probabilistic applications of our categorical framework: Re- 
cently, a sizeable body of work in categorical probability theory has been devel- 
oped in terms of copy-delete and Markov categories. A Markov category [17] is 
a copy-delete category (C,@, I) where every morphism f : X > Y is discardable 
in the sense that discardy o f = discard y. Classic examples of Markov categories 
are the category FinStoch of finite sets and stochastic matrices, and the category 
Stoch of measurable spaces and Markov kernels. Discardability expresses that 
probability measures are normalized (integrate to 1). Markov categories provide 
a natural semantic domain for probabilistic programs [30]. 

In this section, we will focus on Gaussian probability, by which we mean the 
study of multivariate normal (Gaussian) distributions and affine-linear maps. 
This is a small but expressive fragment of probability, which suffices for a range of 
interesting application from linear regression and Gaussian processes to Kalman 
filters. The univariate normal distribution M (u, 07) is defined on R via the den- 
sity function 


Multivariate Gaussian distributions are easiest described as the laws of ran- 
dom vectors A-X +p where A € R"** and X1,..., Xp ~ N(0, 1) are independent 
variables. The law is fully characterized by the mean p and the covariance ma- 
trix X = AA’. Conversely, for every vector u € R” and positive semidefinite 
matrix X € R"*", there exists a unique Gaussian law N (u, X). If X ~ N (u, X) 
and Y ~ N(w’, X') are independent then X +Y ~ N(w+ yp’, X + X') and 
AX ~ N(Au, AXAT). Gaussians are self-conjugate: If (X,Y) are jointly Gaus- 
sian, then so is the conditional distribution X|Y = y for any constant y € R*. 


If the covariance matrix X is positive definite, then the Gaussian has a density 
with respect to the Lebesgue measure on R” given by 


1 


= ex ae T tL 
Fle) = ees eww (Fle -Wae W 


where 2 = X7! is known as the precision matrix. This suggests two equivalent 
representations of Gaussians with different advantages (e.g. [20, 31]): 


— In covariance representation X, pushforwards (addition, marginalization) 
are easy to compute. Conditioning requires solving an optimization problem 

— In precision representation (2, conditioning is straightfoward. Pushforwards 
require solving an optimization problem. 


If X is singular, the Gaussian distribution is only supported on the affine 
subspace u + S where S = im(X). In that case, the distribution has a density 
only with respect to the Lebesgue measure on S. This variability of base measure 
makes it complicated to work with densities, and by extension the precision 
representation. 


180 D. Stein and R. Samuelson 


The situation becomes clearer if we represent Gaussians by the quadratic 
functions induced by their covariance and precision matrices. These functions 
are convex (concave), and turn out to be adjoints of each other. This explains 
the duality of the two representations, and paves the way for generalizations 
of Gaussian probability like improper priors [31] which correspond to partial 
quadratic functions (§5.2). 


5.1 Embedding Gaussians in Bifunctions 


We now give a categorical account of Gaussian probability (in covariance rep- 
resentation). A Gaussian morphism R™ — R” is a stochastic map of the form 
cry Az +N (u, X), that is a linear map with Gaussian noise. 


Definition 6 ([17, §6]). The Markov prop Gauss is given as follows 
1. objects are the spaces R”, and R™ @ R” = R™*” 
2. morphisms R™ — R” are tuples (A, u, X) with A € R°*™, u E€ R” and 


X € R"*” positive semidefinite 
8. composition and tensor are given by the formulas 


(A, u, X) o (B, u’, X") = (AB, p + Ap’, X + AX' A”) 
(A,n, X) @ (B, w, X')= (ASB pep’, X 8X") 


where ® is block-diagonal composition. 
4. the copy-delete structure is given by the linear maps A,! 


We have a Markov functor Gauss —> Stoch which sends R” to the measurable 
space (R”, Borel(R”)) and assigns (A, u, X) to the probability kernel given by 
zr N (Az+u, X). Functoriality expresses that the formulas of Definition 6 agree 
with composition of Markov kernels given by integration of measures. Our main 
theorem shows that, surprisingly, the representation of Gaussians by quadratic 
functions is also functorial, i.e. we have an embedding Gauss —> CxBiFn. 


Theorem 5. We have functors of copy-delete categories in a duality situation 
Gauss 


logpdf cgf 


(CvBiFn°P, e) 7 Cy * (CxBiFn, o) 


The functors are defined as follows: Let f = (A, u, X) € Gauss(R™, R”), and 
define bifunctions 


: 1 
logpdf ; : R” — R™, logpdf;(y, £) = -3 Xz) — {z€ S} 


ae 1 
cgfp:R™ = R", caf s(a,y) = 30 Dy) + (uy) + {le = AT yh 


where z = y — (Ax + u), S =im(2) and X7 denotes any generalized inverse of 
a. 
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Proof (Sketch). Functoriality of cgf follows from a straightforward computation, 
and one can check that cgf; = logpdf, using the formula of [27, p. 109]. Func- 
toriality for logpdf then follows from Theorem 1. The full proof is elaborated in 
the extended version of this paper [32]. 


The value logpdf p(y, x) is indeed the conditional log-probability (4) minus a 
scalar. The name cgf is short for cumulant-generating function, which we elabo- 
rate in §6.2. For now, we can see cgf as a generalized covariance representation. 


5.2 Outlook: Gaussian Relations 


Measure-theoretically, there is no uniform probability distribution over the real 
line. Such a distribution, if it existed, would be useful to model complete ab- 
sence of information about a point X — in Bayesian inference, this is called an 
uninformative prior. Intuitively, such a distribution should have density 1, but 
this would not integrate to 1. On the other hand, a formal logdensity of 0 makes 
sense — this is simply the indicator function of the full subset R C R. 

An extended Gaussian distribution, as described in [31], is a formal sum 
N (u, ©’) + D of a Gaussian distribution and a vector subspace D C R”, called a 
fibre, thereby blending relational and probabilistic nondeterminism. Such enti- 
ties were considered by Willems in the control theory literature, under the name 
of linear open stochastic systems [35, 36]; he identifies them with Gaussian distri- 
butions on the quotient space R"/D. A categorical account based on decorated 
cospans is developed in [31]. 


It is straightforward to embed extended Gaussian distributions into convex 
bifunctions, by taking the sum of the interpretations from Theorems 4 and 5. 
The distribution Y = N (u, X) + D has a convex interpretation given by 


caf y(2) = F(x, Zn) + (u,2) + {2 € DHH 
Functions of this form are partial convex quadratic functions, which are known 
to form a well-behaved class of convex functions (see appendix of the extended 
version [32]). The theory of such functions can be understood as an extension 
of Gaussian probability with relational nondeterminism and conditioning, which 
we term Gaussian relations. In Gaussian relations, we achieve full symmetry 
between covariance and density representation (that is, there exists a color-swap 
symmetry). 

Partiality is necessary to be able to interpret all generators of (1); on the 
upside, the presence of partiality makes conditioning a first-class operation: For 
example, if f : R? — T is the joint logdensity of Gaussian variables (X,Y), then 
conditioning on Y = 0 is the same as computing the bifunction composite with 
the zero map, which is a simple restriction of logdensity fx)y—o(x) = f(x, 0). On 
the other hand, conditioning in the covariance representation f* requires solving 
the infimization problem inf,. {f*(«*,y*)}. Graphically, we have 


vs. f* 


of © 
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6 A Wider Perspective 


The example of Gaussian probability was particular situation in which we could 
map probabilistic concepts to concepts of convex analysis in a functorial way. In 
this section, we will take an even wider perspective and view convex bifunctions 
as a categorical model of probability on its own. We will then point out known 
connections between probability theory and convex analysis, such as the Laplace 
approximation and cumulant generating functions. 


6.1 The Laplace Approximation 


For every copy-delete category C, the subcategory of discardable morphisms is a 
Markov category, and can therefore be seen as a generalized model of probability 
theory. We investigate this notion for categories of bifunctions. 


Proposition 7. Let F: R” — R”,G : R” — R” be bifunctions. Then 


1. F is discardable in (CxBiFn,e) if Yx, inf} F(x,y) =0 
2. G is discardable in (CvBiFn°?,o) if Yx, G(0, x) = {|x = 0} 


and the adjoint (—)* defines a bijection between the two. 
Proof. Direct calculation. 


The embedding of Theorem 5 takes values in discardable bifunctions and hence 
preserve Markov structure. Functoriality means that the composition of Gaus- 
sians (integration) and the composition of bifunctions (optimization) coincide. 
For general probability distributions, this will no longer be the case. We can how- 
ever understand bifunction composition as an approximation of ordinary proba- 
bility theory under the so-called Laplace approximation. In its simplest instance, 
Laplace’s method (or method of steepest ascent) is a method to approximate 
certain integrals by finding the maxima of its integrand (e.g. [34]) 


[ooo exp (sup {(c, x) — so) for n + œ 


A wide class of commonly used probability distributions is log-concave, in- 
cluding Gaussian, Laplace, Dirichlet, exponential and uniform distributions. 
Laplace’s approximation (e.g. [22, §27]) is a way of approximating such distri- 
butions around their mode xg by a normal distribution, as the Taylor expansion 
of their logdensity resembles a Gaussian one 


h(a) ~ h(ao) + Sh" (ao) (a — 20)? 


We can attempt to reduce questions about such distributions to mode-finding 
(maximization). The Laplace approximation is fundamental in many applica- 
tions such as neuroscience [15,16] and has been generalized to a large body of 
literature on so-called saddle-point methods [10, 24]. The existence of the functor 
from Gaussians to bifunctions expresses that, as desired, the Laplace approxima- 
tion is exact on Gaussians. We give an example of the approximation not being 
exact (ironically) on Laplacian distributions. 


A Compositional Framework for Convex Analysis 183 


Example 7. The standard Laplacian distribution has the density function f(x) = 
4 exp(|a|) on the real line. The logpdf h(x) = |x| is a convex function whose 
convex conjugate is given by h*(x*) = {||a*| < 1]} (see Example 2). The latter 
function is idempotent under addition, and conversely h O h = h, so h is idem- 
potent under infimal convolution. In contrast, the density f(a) is not idempotent 
under integral convolution: The sum of independent standard Laplacians is not 
itself Laplacian. 


6.2 Convex Analysis in Probability Theory 


For a random variable X on R”, the moment generating function Mx is defined 
by the following expectation (provided that it exists) Mx(x*) = E[e‘**)]. The 
cumulant-generating function is defined as its logarithm cx (a*) = log Mx(a*). 
The function cx is always convex. The cumulant-generating function of a mul- 
tivariate Gaussian X ~ N (u, X) is precisely 


ex (at) = 5 ("D2") + (e*n) (5) 


which explains our choice of the convex bifunction cgf associated to a Gaus- 
sian morphism in Theorem 5. The notion of cumulant-generating function has a 
central place in the study of exponential families. 

It is a particular fact about Gaussians that the cumulant-generating function 
is the convex conjugate of the logdensity. In the general case, the convex conju- 
gate c%(x) does have a probabilistic interpretations as a so called-rate function 
in large deviations theory (Cramér’s theorem, [13]). It has also been used to 
formulate a variational principle [38]. 


6.3 Idempotent Mathematics 


We zoom out to an even wider perspective: This subsection briefly outlines some 
further background of the connections between convex and probabilistic world: 
The logarithm of base t < 1 defines an isomorphism of semirings ([0, 00), x, +) > 
(R U {+00},+,@¢) where P; is x Di y = log,(t® + t”). In the ‘tropical limit’ 
tN 0, we have x @; y ~ min(x,y), so we can consider working in the semiring 
(R, +, min) as a limit or deformation of the usual operations on the reals. The 
semiring R is idempotent, meaning z @ x = min(x, x) = x, hence this field of 
study is also known as idempotent mathematics [26], and the limiting procedure 
has been called Maslov dequantization [21]. Our definition of convex bifunctions 
in terms of the idempotent semiring R thus carries a strong flavor of idempotent 
mathematics. 

Idempotent analogues of measure theory are discussed in [26,21], and many 
theorems in classical probability theory are mirrored by theorems of idempotent 
probability theory. For example, the idempotent analogue of integration is in- 
fimization; under this view, the tropical analogue of the Laplace transform (cf. 
moment-generating function) is the Legendre transform [21, §7] 


feted int (l,a) + a) 
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which explains the appearance of the cumulant-generating function in our work. 
Theorem 5 means that for Gaussians, it makes no difference whether we work in 
the real-analytic or idempotent world. Idempotent Gaussians have been defined 
in [26, 1.11.10] using the same formula (5). 


7 Related and Future Work 


We have described categories of bifunctions as a compositional setting for con- 
vex analysis which subsumes a variety of formalisms like linear functions and 
relations, as well as convex optimization problems, and has a rich duality theory 
and an elegant graphical language. We have then explored connections between 
convex analysis and probability theory, and showed that Gaussian probability 
can be equivalently described in a measure-theoretic and a convex-analytic lan- 
guage. The equivalence of these two perspectives is elegantly formalized as a 
structure-preserving functor between copy-delete categories. It will be interest- 
ing to see how this approach can be generalized to larger classes of distributions 
such as exponential families. 

Concurrently to our work, the categorical structure of convex bifunctions 
has been exploited by [19] to compositionally build up objective functions for 
MPC in control theory. That work does not explore Legendre duality and the 
connections with categorical models of probability theory. The language of props 
has a history of applications in engineering [2,1,7], and our work was directly 
inspired by the semantics of probabilistic programming [33, 30]. 

A starting point for future work is to flesh out the outlook given in §5.2, 
that is to define a hypergraph category of partial quadratic convex functions, 
which generalizes Gaussian and extended Gaussian probability. It is also inter- 
esting to give a presentation for this prop in the style of [25]: We believe that 
this is achieved by the addition of a single generator v : I — R to graphi- 
cal affine algebra [6] which represents the quadratic function f(z) = x°, and 
that its equational theory is essentially given by invariance under the orthogonal 
groups O(n). A similar equational theory has been attempted in [33] though 
no completeness has been proven. Diagrammatic presentations of concepts from 
geometry and optimization such as polyhedral algebra and Farkas lemma have 
been given in [4,5]. 

We realize that the dependence on regularity assumptions (the caveat of §2) 
makes general theorems about categories of bifunctions like Theorem 1 somewhat 
awkward to state. We still believe that using a general categorical language is a 
useful way of structuring the field and making connections, but see the following 
avenues of improving the technical situation 


1. Identifying specific, well-behaved subcategories of bifunctions (such as con- 
vex relations, (partial) linear and (partial) quadratic functions) on which 
everything behaves as desired. This was pursued in §4 and §5. 

2. The Legendre-Fenchel transform has been phrased in terms of enriched ad- 
junctions in [37]. It stands to hope that developing this enriched-categorical 
approach may take care of some regularity conditions in a systematic way. 
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Abstract. Nondeterministic Discounted-Sum Automata (NDAs) are non- 
deterministic finite automata equipped with a discounting factor A > 1, 
and whose transitions are labelled by weights. The value of a run of an 
NDA is the discounted sum of the edge weights, where the i-th weight is 
divided by à. NDAs are a useful tool for modelling systems where the 
values of future events are less influential than immediate ones. 

While several problems are undecidable or open for NDA, their deter- 
ministic fragment (DDA) admits more tractable algorithms. Therefore, 
determinization of NDAs (i.e., deciding if an NDA has a functionally- 
equivalent DDA) is desirable. 

Previous works establish that when A € N, then every complete NDA, 
namely an NDA whose states are all accepting and its transition function 
is complete, is determinizable. This, however, no longer holds when the 
completeness assumption is dropped. 

We show that the problem of whether an NDA has an equivalent DDA is 
decidable when A € N (in particular, it is in EXPSPACE and is PSPACE —hard). 


Keywords: Discounted Sum Automata - Determinization - Quantita- 
tive Automata 


1 Introduction 


Traditional methods of modelling systems rely on Boolean automata, where ev- 
ery word is assigned a Boolean value (i.e., accepted or rejected). This setting is 
often generalized into a richer, quantitative one, where every word is assigned a 
numerical value, and thus the Boolean concept of a language, i.e., a set of words, 
is lifted to a more general function, namely a function from words to values. 

A particular instance of quantitative automata is that of discounted-sum 
automata. There, the weight function sums the weights along the run, but dis- 
counts the future. Discounting as a general notion is a well studied concept in 
game theory and various social choice models [9]. Computational models with 
discounting, such as discounted-payoff games [21, 3,1], discounted-sum Markov 
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Decision Processes [17, 19, 14] and discounted-sum automata |15, 12,13, 11], are 
therefore useful to model settings where the far future has less influence than 
the immediate future. 

In this work we focus on non-deterministic discounted-sum automata (NDAs). 
An NDA is a quantitative automaton equipped with a discounting factor > 1. 
The value of a run is the discounted sum of the transitions along the run, where 
the value of transition i is divided by àt. The value of a word is then the value 
of the minimal accepting run on it. We also allow final weights that are added 
to the run at its end (with appropriate discounting). 

Unlike Boolean automata, NDAs are strictly more expressive than their de- 
terministic counterpart (DDAs) [11]. In particular, certain decision problems for 
NDAs are undecidable, but become decidable for DDAs [5]. There is, however, 
a subclass of NDAs that always admit an equivalent DDA: the complete integral 
NDAs [6]. An automaton is complete if its transition function is total and all 
its states are accepting with final weight 0. This means that runs never “die”, 
and that all runs are accepting. An NDA is integral if its discounting factor 
is an integer. It is further shown in [6] that if the completeness requirement is 
removed then for every discounting factor there is an integral NDA that is not 
determinizable. 

The existence of NDAs that are not determinizable implies that the deter- 
minization problem is not trivial. However, its decidability and complexity have 
not been studied. In this work, we show that determinization of integral NDAs 
is decidable. Specifically, we show that determinization is in EXPSPACE and is 
PSPACE — hard. 


Example 1. We demonstrate the determinization problem, as well as some intri- 
cacies involved in its analysis. Consider the NDA in Figure la. Intuitively, the 
NDA either reads only a’s, or reads a word of the form a*b. However, it guesses 
in go whether it is going to read many a’s, in which case it may be worthwhile 
incurring weight 3 to q2 in order to read the remaining a’s at cost 0. 


a3 PX 40 T 


a, 0 a, 2 a, 2 


(a) NDA A. (b) Equivalent DDA for à = 3. 


Fig. 1: The NDA A on the left is determinizable with À = 3, with an equivalent 
DDA depicted on the right. However, A is not determinizable with À = 2. 


We now ask if this NDA has a deterministic equivalent. As it turns out, 
this is dependent on the discounting factor. Indeed, consider the discounting 
factor A = 3, then when reading the word a”, the run that remains in qo has 
weight ae. -3-' = 3—3-* < 3, whereas a run that moves to q2 at step 
j > 0 has weight S27) 2:37 +3- 37i = 3 — 37I +371 = 3. Thus, it is always 
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beneficial to remain in qọ. In this case, we do have a deterministic equivalent, 
depicted in Figure 1b. We remark that the fact that this deterministic equivalent 
is obtained by removing transitions is not a standard behaviour, and typically 
determinization involves a blowup. 

Next, consider the discounting factor A = 2. Similar analysis shows that for 
the word a”, the weight of the run that stays in qo is is 229 VASO 29F 8, 
whereas leaving to state q2 at step 0 yields cost 3, so the latter is preferable for 
large k. Intuitively, this means that nondeterminism is necessary in this setting, 
since the NDA does not “know” whether b will be seen. Indeed, for A = 2 this 
NDA is not determinizable. 

Observe that in the case of A = 2, the two “extreme” runs on a”, namely the 
one that stays in go and the one that immediately leaves to q2, create a “gap” 
between their values that tends toward 1 as k increases. Keeping in mind that 
for large k the transition value is multiplied by 27%, intuitively this gap becomes 
huge. As we show in this work, this concept of gaps exactly characterizes whether 
an NDA can be determinized. 


k 


We remark that for non-integral NDAs, many problems, including the deter- 
minization problem, are open due to number-theoretic difficulties [8]. Therefore, 
it is unlikely that progress is made there, pending breakthroughs in number 
theory. 


Related Work Discounted-sum automata have been studied in various contexts. 
Specifically, certain algorithmic problems for them are still open, and are closely 
related to longstanding open problems [8]. In addition, they are not closed under 
standard Boolean operations [7] (which is often the case in quantitative models, 
due to the “minimum” semantics which conflicts with notions of conjunction). 

Recently, discounted sum automata were also studied in the context of two- 
player games [10]. Of particular interest are “regret-minimizing strategies”, where 
the concept of regret minimization is closely related to determinization of au- 
tomata [16]. 

An extension of discounted-sum automata to multiple discounting factors 
(NMDAs) was studied in [4, 5], where NMDAs are NDAs where every transition is 
allowed a different discounting factor. NMDAs are generally non-determinizable, 
but imposing certain restrictions on the choice of discounting factors can ensure 
determinizability [4]. We remark that the study of NMDAs is still only with 
respect to complete automata. 

Determinization of other quantitative models has also received some attention 
in recent years. A major open problem is the decidability of determinization for 
weighted automata over the tropical semiring (for some subclasses it is known to 
be decidable [20, 18]). Interestingly, a tropical weighted automaton can be seen 
as the “limit” of NDAs where A — 1. This, however, does not seem to help in 
resolving the decidability of the former. 

In [2], the determinization problem for one-counter nets (OCNs) is studied. 
OCNs are automata equipped with a counter that cannot decrease below zero. 
They can be thought of as pushdown automata with a singleton stack alphabet. 


194 S. Almagor and N. Dafni 


Most notions of determinizability introduced in [2] are undecidable, with one 
case being open (and seemingly related to the setting of weighted automata). 
Due to space constraints, some proofs appear in the full version. 


2 Preliminaries 


A nondeterministic integral discounted-sum automaton (NDA) is a tuple A = 
(X, Q, Qo, a, 6, val, fval, A), where X is a finite alphabet, Q is a finite set of 
states, Qo C Q is a set of initial states, a C Q is a set of accepting states, 
6CQx » x Q is a transition relation, val : 6 — Z is a weight function that 
assigns to each transition (p,o,q) € ô a weight val((p,o,q)) € Z, fval :a > Z 
is a final weight function that assigns a final weight! to every accepting state, 
and 1 < à € N is an integer discounting factor. 

The existence of a transition (p, o,q) E€ 6 means that when A is in state p and 
reads the letter ø it can move to state q. If there exists q such that (p,o,q) € 6, 
we say that p has a o-transition. If p does not have a o-transition, that means 
that when in state p and reading the letter o, A’s run cannot continue. 

Consider a word w = wi-::Wn E X*. A run of A on w is a sequence of 
states p = /0,/P1;---,Pn such that po € Qo and (p;-1, Wi, pi) € ô for every 
1<i<n. The run is accepting if pn € a. The weight of p is the discounted sum 
val(p) = Xico AT’val (pi, wi41, Pitt): 

The value of w by A, denoted A*(w), is min{val(p) + A™”fval(pn) | p = 
P0;-++;Pn iS an accepting run on w}, that is, the minimal weight of a run on w 
including final weights, or co if no such run exists. Two NDAs A, B are equivalent 
if A*(w) = B* (w) for every w € X*. 

We say that A is deterministic (DDA, for short) if |Qo| = 1 and {q € 
Q\(p,0,¢) € 6} < 1 for every p € Q,o € X. Note that if A is deterministic 
then for every word there is at most one run starting in each state. For a DDA 
we define the partial function 6* : Q x X* — Q such that 6*(q,w) is the final 
state in the run on w starting in q, if such a run exists. We say that an NDA A 
is determinizable if it has an equivalent DDA. 

It will also be useful to consider non-accepting runs and runs that start and 
end in specific states of A. For sets of states P,P’ C Q we define Ajp_, p(w) 
to be the weight of a minimal run of A on w from some state in P to some 
state in P’. Similarly, Ajp—,, p(w) is the minimal weight of an accepting run 
including final weights. When P or P’ is a singleton {p} we omit the parenthesis. 
When P = Qo and P’ = Q (or a, for the setting of including final weights) we 
omit the sets and write e.g., A(w) instead of Ajg,_,qj(w), and A* (w) instead of 
Alao—aj(w). Under these notations, if a run does not exist, the assigned value 
is oo. For the remainder of the paper, fix an integral NDA A. 


1 In some works, the weights are assumed to be rational. For determinizability we 
can assume all weights are integers, since we can always multiply every weight by a 
common denominator. 
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3 Gaps and Separation of Runs 


In this section we lay down the basic definitions we use throughout the paper, 
concerning the ways several runs on the same word accumulate different weights. 

Denote by m 4 the maximal absolute value of a weight of a transition or a final 
weight in A. Recall that the geometric sum (for À > 1) satisfies 77° A7* = zà- 


Therefore, sym A is an upper bound on |val(p)| for any run p. Indeed, we have 


|val(p0,---;Pn)| = |Z; A ‘val (pi, wit, Pi41)| < Dip A “ma < ama 
Clearly, the same bound holds when including final weights. 

Let M = 2,4,mz, then for every two runs p',p? we have |val(p!) — 
val(p”)| < àma — (zma) = M. The constant M is central in our study 
of gaps between runs. 

Consider a word w € X*. The run attaining the minimal value A*(w) might 
not be minimal while reading prefixes of w. The gap between the value of an 
eventually-minimal run and minimal runs on prefixes of w is central to char- 
acterizing determinizability of NDAs [6]. This gap is captured by the following 
definition. 


Definition 1 (Recoverable gap). Consider words w,z E€ X* and states 
du, E Q. the tuple (w,qu,qm) is called a recoverable gap with respect to z, 
or simply a recoverable gap, if the following hold: 


1. Alao ai] (w) < Alao qu] (W), and 
2. Ajao—qu}(w) + A Ales cel (z) = A* (wz) < oo. 


Intuitively, in a recoverable gap (w, qu, qı) there are runs pı and p2 of A on 
w that end in qu and qı, respectively, where pı attains a higher value than p2, 
but there is a suffix z that “recovers” this gap: when reading z from qu starting 
with weight val(p1), the resulting minimal run including final weight attains the 
minimal value of a run of A on w- z. This is depicted in Figure 2. 

For a recoverable gap (w, qu, q1) we define gap(w, qu, q1) = AI”! (Alaosau](w)— 
Algo a](w)). The normalizing factor Al”! eliminates the effect of the length of 
w on the gap, allowing us to study gaps independently of the length of their 
corresponding words. 

We say that A has finitely/infinitely many recoverable gaps if the set 
{gap(w,du,q) | w E€ ©*,qu,q E Q} is finite/infinite, respectively. Note that 
since A is integral, AYI (Aig, .q,](w) — Algo a] (w)) is always an integer and so 
the existence of infinitely many recoverable gaps is equivalent to the existence 
of unboundedly large recoverable gaps. 

While gaps refer to two distinct runs, we sometimes need a more global view 
of gaps. To this end, we lift the definition to all the reachable states, as follows. 


Definition 2 (n-separation). For a word w and n € N, we say that w has the 
n-separation property if there exists a partition of Q into two non-empty sets of 
states U, L such that the following holds: 
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Fig. 2: The run pj, ending in state qı, is the minimal run of A on w. The higher 
run pj is the minimal run on w that ends in state qu, thus creating a gap 
between qu and qı. However, the concatenation p% - p% is the minimal run on the 
concatenated word wz, while the concatenation p} - pb, where p% is the minimal 
run on z starting in qı, is not smaller. Therefore, the gap is recoverable. Note 
that here the final weights are zero. 


1. For every qu E€ U and qı € L, A! (Ajagqu](W) — Alan sqj(w)) > n. 
2. There exist qu E U and z E€ X* such that for every qı E€ L, (w,du,q) is a 
recoverable gap with respect to z. 


We sometimes explicitly specify that w has the n-separation property with re- 
spect to (U, L,qu), or with respect to (U, L, qu,z). If there exists w with the 
n-separation property, we say that A has the n-separation property. 


See Figure 3 for a depiction of n-separation. 


4 Determinizability of Integral NDAs is Decidable — 
Proof Overview 


Recall that our goal is to show the decidability of the determinization problem. 

As showed in [6], determinizability is closely related to recoverable gaps. More 
precisely, a DDA D that “attempts” to be equivalent to A must keep track of all 
the relevant runs of A. If two runs end in the same state, it is clearly enough to 
track only the minimal one. However, this may still require keeping track of runs 
that attain unboundedly high values (when normalized). Therefore, in order for 
D to be finite, it must discard information on runs that get too high. The main 
issue is whether we can give a bound above which runs are no longer relevant. 

For complete integral NDAs, there are always finitely many recoverable gaps, 
and this is used to show that complete NDAs are always determinizable [6]. For 
a general integral NDA A, we similarly show in Section 5 that if there are only 
finitely many recoverable gaps, then A is determinizable. 

There are now two main challenges: First, to show that if A has infinitely 
many recoverable gaps, then it is not determinizable, and second, that it is 
decidable whether A has finitely many recoverable gaps. 
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w z 


Fig. 3: Depicted are minimal runs of an NDA on a word w that end in each of 
four states, q1, q2, q3,q4, and minimal runs on z starting in each of them. The 
run from qı (lowest) “gets stuck”, i.e., such a run from qı on z does not exist. 
The states are partitioned into two sets L = {q1,q2} and U = {q3,qa}, with a 
gap larger than n between them after reading w; additionally, one of the upper 
runs then becomes minimal after reading z, since each of the lower runs either 
ends higher or “gets stuck”. This means that the word w has the n-separation 
property with respect to (U, L, qs, z). 


We start by showing in Section 6.1 that we can compute a bound N such 
that A has infinitely many recoverable gaps if and only if it has a recoverable 
gap larger than M. Next, in Section 6.2, we show that the existence of a gap 
larger than M is also equivalent to some word having the N’-separation property. 


We then turn to exhibit a small-model property on witnesses for \’-separation. 
Specifically, we show in Section 7 that if there exist w, z such that w has the M- 
separation property with respect to (U, L,qu,z), then we can bound the length 
of the shortest w, z. 


Using the above, we obtain the decidability of whether A has infinitely many 
recoverable gaps. In addition, we use these results to prove (in Lemma 11) that 
if A has infinitely many recoverable gaps, then it is not determinizable. This 
allows us to conclude the decidability of determinization in Theorem 1. 


Conceptually, our approach can be viewed as a “standard” one when treat- 
ing determinization of quantitative models, in the sense that considering gaps 
between runs generally characterizes when a deterministic equivalent exists [16, 
2|. The crux is showing that this condition is decidable. To this end, our work 
greatly differs from other works on weighted automata in that we establish the 
decidability of the condition. Technically, this involves careful analysis of the 
behaviours of runs under discounting. 
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5 Finitely Many Recoverable Gaps Imply 
Determinizability 


The main result of this section is an adaptation of the determinization techniques 
in [6] from complete to general automata. While the construction itself is similar, 
the correctness proof requires finer analysis. We remark that in the case that A 
is a complete NDA and all final weights are zero, the construction obtains a 
complete DDA with all final weights zero, thus generalizing the result in [6]. 


Lemma 1. If an NDA A has finitely many recoverable gaps, then it is deter- 
minizable. 


Proof. Let A = (X, Q, Qo, a, ô, val, fval, à) be an NDA with finitely many re- 
coverable gaps. We construct a DDA D = (X, Qp, {vo}, ap, Op, valp, fvalp, ÀA) 
that is equivalent to A. 

Since A has finitely many recoverable gaps, there exists a bound B € N on the 
size of those gaps. The states of D are then Qp = {0,...,B,oo}®. Intuitively, 
a run of D tracks, for each q € Q, the gap between the minimal run of A on 
w ending in q and the minimal run on w overall. When this gap becomes too 
large to be recoverable, the states corresponding to the higher run are assigned 
oo. For v € Qp and q € Q, we denote by (vq) the entry in v corresponding to q. 


The initial state is therefore (vo)q = US Qo assigning for each q € Q the 
œ q ÉQo 
weight of the minimal run of A on the empty word ending in q. 

We now turn to define ôp. Intuitively, when taking a transition, D D first 
updates the vector entry of every state with the value of the minimal run on 
the new word ending in it, using the values specified in the last vector. Then, 
if the minimal entry is not 0, the entries are shifted so that it becomes 0, and 
the value subtracted from every entry is assigned to the transition weight. Fi- 
nally, the entries are all multiplied by A to account for the word length. Thus, 
the actual value of the minimal run is exactly the value attained by D, and 
the vector entries correctly represent the normalized gaps. The construction is 
demonstrated in Figure 4. Formally: 


— For every v € Qp, and for every o € X such that there exists q E€ Q with 
vq < œ and q has a o-transition, define u € {0,...,B,0o}® as follows. 

e Define the intermediate vector u’: For every q € Q, u} = mingeg (vg + 
val(q',o,q)), where val(q’,o,q) is regarded as œ if (q’,0,q) ¢ ô. 

e Define r = mingeg ug the offset of the vector from 0. Note that r is 
finite due to the requirement that there exists q E€ Q with vg < oo and q 
has a o-transition. 

Alu =r) Altuy—r)<B 


e For ever E Q, Ug = . 
VES Gea oo otherwise 

Where œ is handled using the standard semantics. Note that u € {0,...,B,oo}® 
as A is integral. The manipulations done on the intermediate vector uj 


when defining u, should be viewed as normalization — first subtracting r 
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so that the gap represented by uy is with respect to the minimal run overall 
over w; then multiplying by A to account for the length of w. Note that 
the subtraction of r also implies that mingeg Ug = 0, as is expected since 
mingeg Algoa (w) = A(w). 

— We now introduce the transition (v, o, u) € dp. 

— We set valp(v,o,u) = r. This can be viewed, together with the subtraction 
of r from every entry of u’, as transferring the weight from each entry of u’ 
to the transition. 


(a) an NDA A. (b) An equivalent DDA D. 


Fig. 4: An example of an NDA A (on the left) and the resulting DDA D (on the 
right), with A = 2. The name of each state of D corresponds to a vector whose 
first entry tracks gg and the second qı. We demonstrate the construction using 
the a-transition from (2,0) to itself. First we construct the intermediate vector 
u: (u')gg = min(2 + val(go, a, go), 0 + val(qi,a,q0)) = min(2+ 1,0 +œ) = 3 
and (u’)¢, = min(2 + val(qo, a, q1),0 + val(q, a, q1)) = min(2 + 0,0+ œœ) = 2, 
and so u’ = (3,2). We then have r = 2, which is assigned to the weight of the 
transition, and u = 2(3 — 2,2 — 2) = (2,0). 


We next define ap and fvalp. We set ap to include every vector v such 
that vg < oo for some q € a. We note that the construction can be viewed as 
a generalization of the standard subset construction, where for a vector v, the 
states q that satisfy vu, < oo represent the states that can be reached by A when 
reading w, ignoring those states whose gap is unrecoverable. For v € ap, we 
set fvalp(v) = mingea(v, + fval(q)). Figure 4 depicts an example for an NDA 
and the DDA constructed from it (with no final weights). Note that we do not 
yet actually provide an algorithm for constructing D from A, since that requires 
computing B. 

The correctness of this construction is proved in the full version. 


6 Recoverable Gaps and n-separation 


6.1 A Large Gap is Equivalent to Infinitely Many Gaps 


In this section we show that the existence of infinitely many recoverable gaps is 
characterized by the existence of a (computable) large-enough recoverable gap. 
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Consider a run p = pp... Pn, and recall that val(p) is the weight of p and 
that M = 2,4,m A, where ma is the maximal absolute value of a weight of a 
transition or a final weight in A. We denote by I'(p) = A"val(p) the normalized 
“un-discounted” value of p. For two runs p',p? on the same word w, we are 
interested in the value T (p!) — T (p°), as it captures how far the runs are from 
each other, in the sense of how difficult it is to recover their gap. We claim that 
if two runs get too far from each other, the gap between them from that point on 
can only increase. Intuitively, this is because at each step the value is multiplied 
by A, and so beyond a certain gap size, this multiplication separates the runs 
further even if their added values pull them closer before multiplying by A. 


Lemma 2. Let p! = pj,... ee and p? = pes. PRIE be two runs of A, such 
that Coh,- -p1 )-T p8,- -n p2) > M. Then Dld,- php) -DOR P241)> 


In particular, once the gap between p!, p? is larger than M, concatenating 
any runs to pt, p? can only increase the gap and therefore cannot result in p! 
“bypassing” p?. 


Corollary 1. Let p! = p,..., pi, and p? = p2,...,p% be two runs such that 
val(p') < val(p?). Then for every 0 < i < n, it holds that T (p},...,p}) — 
Tlo.) <S M. 


On the other hand, the gap between two runs cannot increase too much within 
a small number of steps. We capture the contra-positive of this, by showing that 
if two runs reach a large enough gap, then the runs have been far from each 
other for a long suffix. 


Lemma 3. Consider Nsteps, Ngap E N, there exists an effectively computable 
number N such that for any two runs p! = p},...,p} and p? = pẹ,..., p2, if 
I'(p')-I'(p?) >N thenn > nsteps and T(p},... are oes (P0: shi 


Ngap- 


p? 
IF N—Nsteps 


We also need a version of Corollary 1 where the inequality between the 
weights of the runs includes final weights. We claim that concatenating any runs 
to runs that are far from each other cannot result in the lower run "bypassing" 
the upper run, including final weights: 


Lemma 4. Let p”, be two runs of A on w, ending in states qu,qı respec- 
tively, such that T(p”)— I'(p!) > M. Let p“s, p's be accepting runs on z start- 
ing in qu,qı respectively and ending in qu;, qu, respectively. Then val(p“p"f) + 
ATIY] Fual(qu,) > val(p'p!t) + ATIY- fual(q,). 


In particular, once a gap becomes too large, the only way to recover from it 
is if the lower run cannot continue at all. 


Lemma 5. Consider a recoverable gap (w,qu,q.) with respect to z such that 
gap(w, qu, q1) > M, then Aig, +;a](2) < 00 and Ajg a)(2) = œ- 


Determinization of Integral Discounted-Sum Automata is Decidable 201 


Proof. From the second condition in the definition of recoverable gap (Defini- 
tion 1), we have Ajg - ;aj(Z) < œ. Let p”, p' be minimal runs on |w| ending in 
qu, qı respectively. Assume by way of contradiction that Ajq,—, ;a)(2) < 00, that is, 
there exists an accepting run p” on z starting in qı. Let p” be a minimal accept- 
ing run on z starting in qu. Since \!”!(val(p") — val(p')) = gap(w, qu, q1) > M, 
Lemma 4 contradicts the fact that ptp" is a minimal accepting run on wz by 
the definition of recoverable gap. 


We can now prove the main result of this section. 


Lemma 6. There exists an effectively computable number N (depending on A) 
such that A has infinitely many recoverable gaps if and only if there exists a 
recoverable gap (w, qu, qı) such that gap(w, du, q) > N. 


Proof Overview We start with an overview of the more complex direction — 
the existence of a large recoverable gap implies the existence of infinitely many 
recoverable gaps. Assume that (w,qu,q) is a large recoverable gap with respect 
to z. We consider two minimal runs p% , p% on w ending in qu and q, respectively. 
These two runs end “far” from each other, so we can use Lemma 3 to claim that 
for a large enough N, they have already been far from each other for a while. 
Specifically, for the last Nsteps steps the gap between the runs was at least ngap 
for some large Nsteps, Ngap that we choose to fit our needs. 

We now look for two indices i < j among the last nsteps indices of w such that 
pumping the infix of w between 7 and j generates words that induce unboundedly 
large recoverable gaps. To do so, we choose nsteps Such that Q can be partitioned 
into two sets of states — an upper set U and a lower set L, that are far from 
each other and "separate" the runs p%, p% at step i. In particular, pumping the 
infix does not interleave the runs, and maintains the growing gap. The above is 
depicted in Figure 5. We require the following properties: 


1. Every two runs on the prefix w1 ---w,; of w ending in U and in L, respectively, 
that are minimal runs ending in their respective states, are far enough from 
each other to satisfy the condition of Lemma 2; 

2. Every run on w that is minimal among the runs ending in qu has to visit U 
at the ith step; 

3. Every run on w that is minimal among the runs ending in q has to visit L 
at the 7’th step; 


As we show, finding such a partition is possible by choosing ngap = (|Q| — 1)M. 
Next, we show that in fact, U and L induce a certain separation of the run 
trees emanating from them on the pumped words. Specifically, we show that: 


(i) There exist runs of A on the pumped words (denoted w®)) ending in qu, qr- 

(ii) Every run on w®) (ending in any state) that is a prefix of a minimal run 
on wz has to visit U at the tth step. That is, a variant of Condition (2), 
where instead of qu we consider any state p reached after reading w) along 
a minimal run on w) z. 


202 S. Almagor and N. Dafni 


> 
L, a 
Ae gS o 
E 


| 
x (Pumped infix) j Bo 
i 


Bi 


Fig. 5: At step i of A’s run on w, the states are partitioned into an upper set 
U and a lower set L that are separated by a large gap. The runs p%™, p% visit 
U, L respectively, meaning the gap between them can only grow after step i. The 
indices i,7 are chosen such that both runs p%™,p” repeat states and the sets 
of ancestors Anc,(i),Ancg(j) are identical for each q € Q. The state p, which 
is visited after reading a pumped word w) by a minimal run on w®)z, is not 
reachable from L on any of the pumped suffixes. 


(iii) Condition (3) above holds not only for w but for the pumped words w“) as 
well. 


Note that (ii) and (iii) imply that runs on w®) also induce a recoverable gap. 

From this, it follows from Condition 1 and Lemma 2 that the pumped words 
induce unboundedly large recoverable gaps. 

In order to ensure (i), we require Nsteps > |Q|? (which is the length of the 
“large gaps” suffix) such that both runs p™, p% must repeat their pair of respec- 
tive states at some indices 7,7. Consequently, the runs p%™, p% can be pumped 
to achieve the desired runs. 

To ensure (ii), it follows from Corollary 1 and the fact that p™ is a prefix 
of a minimal run on wz that any state p reached along a run on w®)z after 
reading w*) is not reachable from L when reading w;41--- Wiw|, and we want to 
ensure that p is not reachable from L when reading the pumped suffix as well. 
For that, for each state q and for each index of w we consider the set of states 
Anc,(i) from which q is reachable when reading the respective suffix (from index 
i), called the ancestors of q at index i, and it is enough to require that for each 
state q this set is identical for indices 7 and j. This, in turn, requires to increase 
Nsteps by a factor of 21Q1*. Combined with the previous requirement on 7,7, we 
choose Nsteps = IQR, 

Finally, for condition (3) in (iii), we use the fact that there exists a run 
ending in q that visits L at the ith step (namely the pumped run) and apply 
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Corollary 1. Indeed, any run that does not visit L at the tth step must visit 
U instead, and by Corollary 1 and the gap between U and L, it must be larger 
than the run we have that visits L and therefore not minimal. 


Proof (of Lemma 6). Consider runs p4, ... , p} and pĝ, ... , p2. From Lemma 3, we 
can effectively compute N such that if '(pj,...,p;,) —I'(p2,---,p%) >N, then 
2 
n> |Q|?2!2! and T (o, DES Paoa) -T (Pos Aaa P, o]221912) > ((Q|-1)M. 
Assume that (w, qu, 1) is a recoverable gap with respect to z and gap(w, qu, q1) > 
N. Let p™ = p64"... Piu be a run on w ending in qu that is minimal among the 
runs on w ending in qu, and similarly p% = p§)... Piw | for qı. Since these runs are 
minimal runs ending in their respective states, it holds that T (p%) — r (p%) = 
gap(w, qu, q1) > N, and so we have |w| > |Q|?2!@! and Eolia 
T(oĝ, A » Phyl —lq}eatar?) > (IQI _ 1)M. 
For every q E€ Q and 1 <i < |w], let Anc g(t) = {g € Q | Ajg gq (wisi Ww) < 
oo} be the set of ancestors of q at step i, i.e., states from which q is reachable 
when reading the input wj+41,+-- , Wj). By the pigeonhole principle there exist 


|w| — |Q)22'2” < i < j < |w| such that 


1 L 
Phw|jgp2ia2) 


— For every q € Q, Ancg(t) = Anc,(j), 
-= pe = p;" and pe = py 

Write w = 1x bz where b1 = wy +++ Wi, £ = Wi41 ++ wz and By = Wj41-+ + Ww: 
We now turn to show that by pumping x, we can obtain unboundedly large re- 
coverable gaps. 

Let k € N. We can easily show that 6,2" G2 induces unboundedly large gaps 
between qu and qı, but that would not be sufficient: We also need those gaps 
to be recoverable with respect to z, that is, the minimal run on 61g" 682z has to 
visit qu after reading 8,2" 82. However, this is not necessarily true: It can visit a 


different state, and we need to show that that state is also far enough from q;. The 


du du ( Qu du\k plu du qı CIE q\k WU qı 
runs pp"... p;" (Piga P;") Pita- Play ANA PO! «+ - Pj (Pipi PF) jr Pwl 


are runs on 6,2" By ending in qu, q respectively. In particular, since there exists 
arun on z starting in qu, we have that A has a run on §,2" 822. Let p be minimal 
among those runs, and let qmin, be the state p visits after reading By ax* Bo. Let 
piming ® ptk be runs on 6,282 that are minimal among the runs ending in 
dmin, qı respectively. Note that p%inc® can be obtained as a prefix of p, since 
p is minimal. Then we have gap(@12" 82, dmin,,q) = '(pt™»’*) — P'(p%*), and 
it remains to show that T(p%==x:¥) — P'(p%") can get unboundedly large for a 
large enough k. 

We already know that the runs p%!, p% are far enough from each other at 
their tth step to satisfy the condition of Lemma 2, and we want to show that 
the same is true for p?n«*, pt, 

To do so, we intuitively show that after reading 6), the runs p%™, p% have 
become so far apart that they now stem from disjoint sets of states with a large 
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gap between them. Formally, consider the sets 


U' = {q € Q | there exists a run p on w with val(p) = Ajg,+q,](w) and p; = q} 
L' = {q € Q | there exists a run p on w with val(p) = Ajg,+qj(w) and p; = q} 


That is, U’ (resp. L’) is the set of states that appear at step i in a minimal run 
to qu (resp. qı). For every q E€ Q, let v(q) = A Algo] (81) be the “undiscounted” 
value of a minimal run of A on (, ending in q. Then, from Lemma 3 and the 
constants we chose, for every qj, E€ U’, q; E€ L’ we have v(q,)—vlqi) > (|Q|-1)M. 

In particular, there is a partition of Q into two sets U, L such that U’ C 
U,L’ C L and v(p) — v(q) > M for every p € U and q € L. Indeed, otherwise 
the maximal gap between two states is less than (|Q| —1)M. We next show that 


(i) pi™"*"" EU, and (ii) p% € L. 

For (i), we note that Apps qnin,](@42) = 00: Otherwise, since Ajy,,.,,., > fa] (2) < 
oo, there exists an accepting run on wz that visits L after reading 6; and qmin, 
after reading w. By Lemma 4, such a run must be of lower weight than any run 
that visits U after reading 1, in contradiction to the fact that p™ is a prefix of 
a minimal run on wz by the definition of recoverable gap. Since Ancy,,,, (i) = 


AnCgnin, (J), We also have that Alt amin,,] (2 2) = oo. In particular, pees E€ 
Ue 

For (ii), the run p¢' ... of (of, e Pf) ofa -Pie satisfies p? € L (since it 
is in L’), and in particular Ajr_,q,)(v* 2) < oo. By Corollary 1, any run whose 
ith state is in U results in a higher weight than any run whose 2’th state is in 
L, and so since p%* is minimal we have p" € L. 

It remains to show that the runs p=»: p%*, being far from each other at 
the tth step, get unboundedly far from each other as k increases. Let fu, fi : 
{i,i+1,...} — N be defined as follows: 


= fuli) = mingev v(q) 

— For m > i, fu(m+1) = A( fulm) — ma) 
— fili) = maxger v(q) 

— For m >i, film +1) = A(fi(m) + ma) 


Intuitively, fu (resp. fı) represents a lower (resp. upper) bound on the "undis- 
counted" weight of runs visiting U (resp. L) in their i’th step. That is, for every 
m > i we have P(pinine® : pare”) > fulm), and F(p"... p) < film). 
Additionally, fu(i) — fili) > M and so the function f,,(m) — fi(m) increases 
with m. Thus, for every M € N, taking a large enough k, we can obtain 
T (ptm E) — P(pt*) > fullBix" Bel) — fı((81x" 82|) > M. This concludes the 
proof that if A has a large recoverable gap, then it has infinitely many recoverable 
gaps. 

For the converse direction, assume A has infinitely many recoverable gaps. 
Since A is integral, the term A!! (Ajg,-4p](w) —Ajao—+qj(w)) is always an integer, 
therefore infinitely many recoverable gaps imply the existence of unboundedly 
large recoverable gaps, and in particular one larger than N. 
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Remark 1. Following the arguments in the proofs of Lemmas 3 and 6, the number 


N provided i ea 6 equals ylera2ie? (Q| — 1)M — M) + M. We denote 
this value by N. 


6.2 A Large Gap is Equivalent to Separation 


Recall that a gap refers to minimal runs that end in two specific states, but 
ignores the remaining states (to an extent). A more “holistic” view of gaps is 
via separations (Definition 2). In this section we show that the two views are 
equivalent, and that both characterize when A has infinitely many gaps. 


Lemma 7. A has a recoverable gap larger than N if and only if A has the 
N -separation property. 


Proof. Assume that A has a recoverable gap larger than M. By Lemma 6, there 
exist unboundedly large recoverable gaps, and in particular there exists a recov- 
erable gap (w, qu, qı) with gap(w, qu, qi) > (Q| — IN. 

Intuitively, when ordering the states by the weight of the minimal run that 
reaches each state, such a gap implies a gap of at least M between two successive 
states, leading to the desired partition. We then claim that the sets are separated 
by the same suffix z that separates the states from the original gap. 

Write Q = {q1,---, Gq} such that Ajg,oqj(w) < --- < Alay aqiq)](w) (recall 
that if there is no run on w ending in q, then Ajg,-.q\(w) = 00), and let i < iu 
be indices such that q = qi, qu = qi.. Then there exists j € {t,...,tu-1} 
such that Ajgy+qj4:)(w) — Afaosq;(w) > N. Let U = {aj41,---, Q Qi} and 
L = {q,.-.-,q;}. Then for every gj, € U,q} € L we have Nl Algg az] (w) — 
Alo qj] ()) >N. 

Consider z € X* such that the gap (w, qu, qi) is recoverable with respect to 
z. Note that M > M, and so it follows from Lemma 4 that Algi pa] (2) = © 
for every q; € L. Indeed, if A had a run on z starting in q}, concatenating it to 
a minimal run on w ending in q; would result in a run of lower weight than any 
run on wz that visits qu after reading w, contradicting the fact that (w, qu, qi) 
is a recoverable gap. Additionally, it follows from (w, qu, q) being a recoverable 
gap that there exists a minimal run on wz that visits q, after reading w. Then 
(w, du; qj) is a recoverable gap with respect to z, and so w has the N-separation 
property with respect to (U, L, qu, 2). 

For the converse direction, assume that w has the N’-separation property with 
respect to some (U, L, qu, z). In particular, Ajg,—+q,,] (w) +AT Aguja] (z) < œ. 
Let q, € U be such that Ajgo=q, (w) + 7! Ajg: 5 a] (2) is minimal. Let some 
qi € L. Then (w,q/,,q;) is a recoverable gap with respect to z, and it is larger 
than M, as needed. 


7 Bounding the Witnesses for Separation 


In Section 6 we show that A has infinitely many recoverable gaps if and only if 
there exists a word w with the M-separation property. Expanding Definition 2, 
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this happens if and only if there exist a partition of Q into two sets U, L and 
there exist words w, z that “separate” U from L. In this section we can bound 
the length of such minimal w,z. We start with w (see the full version for the 
detailed proof). 


Lemma 8. Let C = 54;(N|Q| + 2m,). Assume that w has the N-separation 
property for some w € X*. Then there exists w' such that w has the N- 
separation property and |w'| < (C +2)!@!, 


Proof (Sketch). Assume that w has the \-separation property with respect to 


(U, L, qu, z). 
We start by using an identical construction to that of Lemma 1, with bound 
C, in order to define a sequence of vectors vo, ...-, Uw] with v; € {0,...,C, oo} @ 


for every 0 < i < |w| that, intuitively, keep track of the runs of A on w, as 
follows. 


0 € 
— For every q € Q set (vo)q = f Qo 
œo otherwise 


— For every i > 0,q € Q let v; q = mingeg((vi-1)q + val(q’, wi, q)), where 
val(q’,o,q) is regarded as œ if (q',0,q) ¢ 6 (the v; q are “intermediate” 
values). 

For every i > 0 let r; = mingeg A é (the r; are the offset of the vector from 


0). 


Alugo ri) Alugo ri) SC 


— For every i > 0,q € Q set (vi)g = ; 
oo otherwise 


Recall that intuitively, (v;) tracks, for each q € Q, the gap between the 
minimal run on w1- -- w; ending in q and the minimal run on this prefix overall. 
When this gap becomes large enough that recovering from it implies the existence 
of N-separation, it is denoted oo. 

Denote the normalized difference A‘(Ajgg+q](wi: +: wi) — A(wi-++w;)) by 
Aq (w). It is easy to show that v; keeps the correct weight of runs whose gap 
from the minimal one remains always under C. However, if a gap of a run goes 
over C but then comes back down, then v; no longer tracks it correctly. To 
account for this, we claim that since w has the M-separation property, for every 
q,i at least one of the following must hold: 


— (v;)q = Ay i(w) Ay i(w) <C 
“2 oe) otherwise 
— There exists i’ < i such that w- wy has the N-separation property. 


That is, either v; tracks the runs correctly, or there is some shorter prefix that 
already has the N-separation property. 

The proof is by induction on 7, with the only problematic case arising when 
(vi-z1) = œ, and so the information about the exact value of the gap repre- 
sented by (vj-1)q’ is gone. We consider the normalization value r; (i.e., the offset 
of the minimal run from 0): if r; is small, then the gap represented by (v;)q is 
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still very large, and we show that marking it as oo is sound. Otherwise, if r; is 
large, then the above gap might indeed be wrongly marked as oo. However, we 
show that in this case, r; is so large that we can actually obtain an N-separation 
property “below” r;, using a shorter witness. More precisely: 


= If @-1)¢ = œ and r; < cè — mg, then since (vi-1) = 00, we have 
(vi) = 00. It remains to show that X'(Ajgy+q(wi +++ wi) — A(wi +++ wi)) > 
C'. Indeed, 


A (Algoa (wi +: wi) — Alw --- wi) 
>A (Agog (Wi + wi) — Alw e wi-1) — (ma + ri) AT?) 
=A (Agong) (Wi Wi-1) — Alwi + wi_-1)) — ri — ma) 
A-1 


>O-(O5 


ma) ma) = MA1C+m,4—ma) > C 


where the first transition follows by observing that when reading w;, in the 
worst case, the weight of a specific run can decrease by A~“~)m_4, and the 
overall weight of the word can increase by \~ Yr; 

-r > or — ma. This is only possible if for every q such that (vi—1)q < 
CA — 2m4 = N|Q|, qı has no w;-transition. Let L” = {q1 € Q | (vi-1)q < 
N|Q|}. Write Q = m,.--, Gq | such that (vi-1)q, < --- < (Vi-1) qq) and so 
L” = {q,---,q\L\}. Since w has the N-separation property, in particular 
A has a run on w and so L” Ç Q. Then, there exists 1 < r < |L”| such 
that Oe a tas — Cae > N. Let U’ = {dr41, eae galt = {q, Swe Ort; 
and note that for every q; E€ L’, q; has no w;-transition. For every qj € 
L', dh E U', we have ATHA Qoa wi tee Wi-1) = Alan sai] (1 oa Wi-1) = 
(vi-1) qi, — (viz1)g > N. Let di, € U’ be such that Alao 34] (wy ae -Wi-1) + 
ATC-D Alg > pa] (wi) is minimal. Then for every q; € L’, (w1 +: wi-1, qs qi) 
is a recoverable gap with respect to w;, and so w,-:-w;—1 has the N- 
separation property with respect to (U’, L’, q}, wi), and we are done. 


Now, it remains to show that if |w| > (C + 2)!@!, there exists w’ such that 
|w’| < |w| and w’ has the N-separation property. 

To this end, we use the induction hypothesis and the pigeonhole principle 
to remove an infix of w, and argue that the resulting word w’ also has the N- 
separation property with respect to some (U’, L’,q/,): Either all of the minimal 
runs ending in the states of L have values far enough (below) of C, in which case 
U', L’ can be chosen to be U, L respectively; or some state of L attains a high 
value, in which case there must be a large gap between two consecutive states of 
L, and the resulting lower set can be chosen as L’. As for q/,, it is simply enough 
to consider the state in U’ that the minimal run on w’z visits after reading w’ 
(see the full version for the details). 


Next, we give a bound on the length of the minimal separating suffix z from 
Definition 2. Recall that by Lemma 5, a large gap can only be recoverable if the 
smaller runs cannot continue at all. Following that, we can now limit the search 
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to suffixes that separate runs in a Boolean sense (i.e., making one accept and 
another reject). This yields a bound from standard arguments about Boolean 
automata, as follows. 


Lemma 9. Consider a word w that has the N-separation property with respect 
to (U,L,qu,z). Then there exists z' such that w has the N-separation property 
with respect to (U, L, qu, 2’) and |z'| < 27181, 


8 Determinizability of Integral NDAs is Decidable 


In this section we establish the decidability of determinization. To this end, we 
start by completing the characterization of determinizable NDAs by means of 
gaps, and then use the results from previous sections to conclude the decidability 
of this characterization. 

Recall that in Lemma 1 we show that finitely many recoverable gaps imply 
determinizability. In this section we show the converse, thus completing the char- 
acterization of determinizable integral NDAs as exactly those that have finitely 
many recoverable gaps. 

We first need the following lemma which is proved in [6, Lemma 5]. 


Lemma 10. Consider an NDA A for which there is an equivalent DDA D. If 
there is a state q of A and words w,w’,z such that: 


— A has runs on w and w' ending in q; 

— gap(w,q,p) # gap(w’,g,p’), where p,p' are the last states of some minimal 
runs of A on w,w’ respectively; 

— both gaps (w,q,p) and (w’,q,p’) are recoverable with respect to z; 


then the runs of D on w and w end in different states. 
We now show the converse of Lemma 1. 


Lemma 11. If an NDA A has infinitely many recoverable gaps, it is not deter- 
minizable. 


Proof. Assume by way of contradiction that A has an equivalent DDA D and 
infinitely many recoverable gaps. For every q € Q, let 


G, = {w | A has a recoverable gap of the form (w, q,p) for some p} 


Since Q is finite and A has infinitely many recoverable gaps, there exists q E€ Q 
such that Gy is infinite. By Lemma 9, there is a finite collection Z of words 
such that every recoverable gap is recoverable with respect to some word in 
Z. Therefore there exist z € Z and an infinite subset G C G, such that for 
every w € G4, the gap (w, q, p) is recoverable with respect to z for some p. By 
Lemma 10, for every two words w, w’ € G}, the runs of D on w and w’ end in 
different states, in contradiction to the fact that D has finitely many states. 
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Consider an NDA A. By Lemmas 1, 6 to 9 and 11 we have that A has an 
equivalent DDA if and only if for every w, z such that |w| < (;4¢(W|Q|+2m.)4 
2)!@! and |z| < 27!@!, it holds that w does not have the N-separation property 
with respect to (U, L, qu, z) for every U, L, qu. Since the latter condition can be 
checked by traversing finitely many words and simulating the runs of A on each 
of them, we can conclude our main result. 


Theorem 1. The problem of whether an integral NDA has a deterministic equiv- 
alent is decidable. 


Remark 2 (Complexity of Determinization). Using the bounds on w, z, one can 
guess w,z on-the-fly, while keeping track of the weights of minimal runs to all 
states, discarding those that go above C as per Lemma 8, to check whether A has 
the M-separation property. Since M is double exponential in the size of A, this 
procedure can be done in NEXPSPACE = EXPSPACE. Thus, determinizability 
is in EXPSPACE. For a lower bound, determinizability is also PSPACE — hard by 
a standard reduction from NFA universality. Tightening this gap is left open. 
Note that for lowering the upper bound, we would need a refined application of 
the pigeonhole principle in Lemma 6, which seems somewhat out of reach for 
the pumping argument. Conversely, for increasing the lower bound, we would 
need to show that using discounting we can somehow force a double-exponential 
blowup in determinization. While this might be within reach, no such example 
are known for e.g., tropical weighted automata, suggesting that this may be very 
difficult. 


Acknowledgments The authors thank Guy Raveh for fruitful discussion re- 
garding Lemma 3. 
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Abstract. We show that the problem of checking if a given nonde- 
terministic parity automaton simulates another given nondeterministic 
parity automaton is NP-hard. We then adapt the techniques used for 
this result to show that the problem of checking history-determinism for 
a given parity automaton is NP-hard. This is an improvement from Ku- 
perberg and Skrzypczak’s previous lower bound of solving parity games 
from 2015. We also show that deciding if Eve wins the one-token game or 
the two-token game of a given parity automaton is NP-hard. Finally, we 
show that the problem of deciding if the language of a nondeterministic 
parity automaton is contained in the language of a history-deterministic 
parity automaton can be solved in quasi-polynomial time. 


1 Introduction 


Deciding language inclusion between two automata is a fundamental problem in 
verification, wherein we ask whether all executions of an implementation satisfy 
a given specification. Unfortunately, the problem of checking language inclusion 
is often computationally hard. For parity automata—which are the focus of this 
paper—it is PSPACE-complete, with PSPACE-hardness already occurring for 
finite state automata [89]. 

On the other hand, simulation is a fundamental behavioural relation between 
two automata [33123], which is a finer relation than language inclusion and is 
easier to check. For parity automata, simulation can be decided in polynomial 
time if the parity indices are fixed; otherwise it is in NP [13]. Note that while 
simulation between two automata is sufficient to guarantee language inclusion, 
it is not necessary. 

For history-deterministic automata, however, the relation of language inclu- 
sion is equivalent to simulation [9J8], thus making them suitable for verification. 
These are nondeterministic automata where the nondeterminism can be resolved 
‘on-the-fly’, just based on the prefix of the word read so far. The definition we 
use here was introduced by Henzinger and Piterman in 2006, where they dubbed 
it ‘good-for-games’ automata, while the term ‘history-determinism’ was coined 
by Colcombet [I5] in the context of regular cost automata. 

History-deterministic parity automata are more succinct than their deter- 
ministic counterparts [28] whilst still maintaining tractability for the problems of 
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verification and synthesis on them [24]28]8]. Consequently, history-deterministic 
parity automata have been the subject of extensive research [28)5)3/37[2[29], 
and has garnered significant attention over the recent years beyond parity au- 
tomata as well, extending to quantitative automata [6[7], infinite state sys- 


tems [21]31)35/10)20), and timed automata [9]. 


Despite these recent research efforts, a significant gap remains in understand- 
ing the complexity of checking whether a given parity automaton is history- 
deterministic. While Henzinger and Piterman have shown an EXPTIME upper 
bound [24], the best lower bound known so far is by Kuperberg and Skrzypczak 
since 2015 [28], who showed that checking for history-determinism is at least as 
hard as finding the winner of a parity game [28]—a problem that can be solved 
in quasi-polynomial time and is in NP N coNP (and even in UP N coUP [26}). 


Kuperberg and Skrzypczak also gave a polynomial-time algorithm to check 
for history-determinism of co-Biichi automata in their work [28]. This was fol- 
lowed by a polynomial time algorithm to check for history-determinism of Biichi 
automata in 2018 by Bagnol and Kuperberg [3], who showed that in order to 
check if a Biichi automaton is history-deterministic, it suffices to find the winner 
of the so-called ‘two-token game’ of the automaton. This connection between 
history-determinism and two-token games was extended in 2020 to co-Biichi au- 
tomata by Boker, Kuperberg, Lehtinen, and Skrzypczak [4]. It is conjectured 
that the winner of the two-token game of a parity automaton characterises its 
history-determinism. While the two-token conjecture is open to date, showing 
this conjecture would imply that one can check history-determinism of a given 
parity automata with a fixed parity index in polynomial time. 


Our contributions. We show that checking for simulation between two parity 
automata is NP-hard when the parity index is not fixed. Since simulation is 
known to be in NP, this establishes the problem to be NP-complete (Theo- 


rem{11). 

An adaptation of our proof of Theorem gives us that checking history- 
determinism for a parity automata is also NP-hard (Theorem [I5}, when the par- 
ity index is not fixed. This is an improvement on Kuperberg and Skrzypczak’s 
result from 2015, which shows that checking history-determinism for parity au- 
tomata is at least as hard as solving parity games [28]. We also show, using the 
same reduction, that checking whether Eve wins the 2-token game (of a given 
parity automaton) is NP-hard, while checking whether Eve wins the 1-token 
game is NP-complete (Theorem [15}. 


As remarked earlier, for history-deterministic parity automata, the relation 
of language inclusion is equivalent to simulation. This gives us an immediate 
NP upper bound for checking language inclusion of a nondeterministic par- 
ity automaton in an HD-parity automata, as was observed by Schewe [87]. We 
show that we can do better, by showing the problem to be decidable in quasi- 
polynomial time (Theorem [20}. 
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Overview of the paper: one reduction for all. The central problem used 
in our reduction is of checking whether Eve wins a 2-D parity game, which is 
known to be NP-complete due to Chatterjee, Henzinger and Piterman [I3]. In 
Section |3| we give a reduction from this problem to checking for simulation be- 
tween two parity automata, thus establishing its NP-hardness (Theorem|11). We 
then show, in Section [4.1] that the problem of checking whether Eve wins a good 
2-D parity games—a technical subclass of 2-D parity games—is also NP-hard. 
In Section we show that modifying the reduction in proof of Theorem 
to take as inputs good 2-D parity games yields NP-hardness for the problems 
of checking history-determinism (Lemma |14) and of checking if Eve wins the 
1-token game or the 2-token game (Theorem|}15). Finally, in Section J] we give a 
quasi-polynomial algorithm to check whether the language of a nondeterministic 
parity automaton is contained in the language of a history-deterministic parity 
automaton (Theorem [20}, by reducing to finding the winner in a parity game. 


2 Preliminaries 


We let N = {0,1,2,---} to be the set of natural numbers, and w to be the 
cardinality of N. We will use fi, j] to denote the set of integers in the interval 
{i i+ 1,...,j} for two natural numbers i,j with i < j, and [j] for the interval 
(0, j]. An alphabet X is a finite set of letters. We use X* and X” to denote the 
set of words with finite and w length over X respectively. We also let € denote 
the unique word of length 0. 


2.1 Parity conditions 


Let G = (V, E) be a (finite or infinite) directed graph equipped with a priority 
function x : E — N that assigns each edge with a natural number, called its 
priority. We say that an infinite path p in G satisfies the y-parity condition if 
the highest priority occurring infinitely often in the path is even. When clear 
from the context, we will drop ‘parity condition’ and instead say that p satisfies 
X- 

A parity condition is easily dualised. Given a priority function y as above, 
consider the priority function x’ := x + 1 that is obtained by increasing all the 
labels by 1. Then, an infinite path satisfies x’ if and only if it does not satisfy x. 


2.2 Parity automata 


A nondeterministic parity automaton A = (Q, X, qo, A, 2) contains a finite di- 
rected graph with edges labelled by letters in X. These edges are called transi- 
tions, which are elements of the set A C Q x X x Q, and the vertices of this 
graph are called states, which are elements of the set Q. 

Each automaton has a designated initial state qo E€ Q, and a priority function 
Q : A — [i,j] which assigns each transition a priority in [i,j], for i < j two 
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natural numbers. For states p,q and an alphabet a € X, we use p —> q to 
denote a transition from p to q on the letter a that has the priority c. 

A run on an infinite word w in © is an infinite path in the automaton, 
starting at the initial state and following transitions that correspond to the 
letters of w in sequence. We say that such a run is accepting if it satisfies the 
N-parity condition, and a word w in X® is accepting if the automaton has an 
accepting run on w. The language of an automaton A, denoted by L(A), is the 
set of words that it accepts. We say that the automaton A recognises a language 
L£ if L(A) = £. A parity automaton A is said to be deterministic if for any given 
state in A and any given letter in X, there is at most one transition from the 
given state on the given letter. 

If A’s priorities are in fi, j], we say that (j —i+1) is the number of priorities 
of A. Since decreasing (or increasing) all of these priorities in the automaton by 2 
does not change the acceptance of a run— and hence a word—in the automaton, 
we will often assume i to be 0 or 1. With this assumption, the interval fi, j] is 
then said to be the parity index of A. A Büchi (resp. co-Btichi) automaton is a 
parity automaton whose parity index is [1,2] (resp. [0, 1]). 


Remark 1. We note that we allow an automatonto be incomplete, i.e. there might 
be letter and state pairs in an automaton such that there are no transitions on 
that letter from that state. 


2.3 Game arenas 


An arena is a directed graph G = (V, E) with vertices partitioned as Vy and Vy 
between two players Adam and Eve respectively. Additionally, a vertex vo € Vy 
is designated as the initial vertex. We say that the set of vertices V3 is owned 
by Eve while the set of vertices Vy is owned by Adam. Additionally, we assume 
that the edges E don’t have both its start and end vertex in Va or W. 

Given an arena as above, a play of this arena is an infinite path starting at 
vo, and is formed as follows. A play starts with a token at the start vertex vo, 
and proceeds for countably infinite rounds. At each round, the player who owns 
the vertex on which the token is currently placed chooses an outgoing edge, and 
the token is moved along this edge to the next vertex for another round of play. 
This creates an infinite path in the arena, which we call a play of G. 

A game G consists of an arena G = (V, E) and a winning condition given by 
a language L C E”. We say that Eve wins a play p in G if p is in L, and Adam 
wins otherwise. A strategy for Eve in such a game G is a function from the set 
of plays that end at an Eve’s vertex to an outgoing edge from that vertex. Such 
an Eve strategy is said to be a winning strategy for Eve if any play that can be 
produced when she plays according to her strategy is winning for Eve. We say 
that Eve wins the game if she has a winning strategy. Winning strategies are 
defined for Adam analogously, and we say that Adam wins the game if he has a 
winning strategy. 

In this paper we will deal with w-regular games. These are games where the 
languages specifying the winning condition are recognised by a parity automata. 
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Such games are known to be determined [82]22], i.e. each game has a winner. 
Two games are equivalent if they have the same winner. 


2.4 Parity games 


A parity game G is played over a finite game arena G = (V, E), with the edges of 
G labelled by a priority function x : E — {0,1,2,--- ,d}. A play p in the arena 
of G is winning for Eve if and only if p satisfies the x-parity condition. 


2.5 Muller conditions and Zielonka trees 


A (C,F)-Muller conditions consists of a finite set of colours C, and a set F 
consisting of subsets of C. An infinite sequence in C” satisfies the (C, F)-Muller 
condition if the set of colours seen infinitely often along the sequence is in F. 

A Muller game G consists of an arena G = (V,£), a colouring function 
m : E — C and a Muller condition (C, F). An infinite play p in G is winning 
for Eve if the set of colours seen infinitely often along the play is in F, and Eve 
wins the Muller game G if she has a winning strategy. 

Every Muller game can be converted to an equivalent parity game, as shown 
by Gurevich and Harrington [22]. We will use the conversion of Dziembowski, 
Jurdzinski, and Walukiewikz that involve Zielonka trees [I6][2], which we define 
below. 


Definition 2 (Zielonka tree). Given a Muller condition (C, F), the Zielonka 
tree of a Muller condition, denoted Zc,F, is a tree whose nodes are labelled by 
subsets of C, and is defined inductively. The root of the tree is labelled by C. 
For a node that is already constructed and labelled with the set X, its children 
are nodes labelled by distinct maximal non-empty subsets X' C X such that 
X EF = X'¢F. If there are no such X’, then the node labelled X is a leaf of 
Zc,F and has no attached children. 


Given a (C, F)-Muller condition, consider the language L C C“ consisting of 
words w that satisfy the (C, F)-Muller condition. The language L is then said 
to be the language of the (C, ¥)-Muller condition, and can be recognised by a 
deterministic parity automaton, whose size depends on the size of the Zielonka 
tree |12]. 


Lemma 3 ([12]). Let (C,F) be a Muller condition with the Zielonka tree Zo. F 
that has n leaves and height h. Then there is a deterministic parity automaton 
DoF that can be constructed in polynomial time such that Do r has n states and 
(h +1) priorities, and accepts the language of the the (C, F)-Muller condition. 


Consider a Muller game G on the arena G = (V, E) with the colouring func- 
tion m : E — C and the Muller condition (C, F). We can then construct an 
equivalent parity game G’ by taking the product of G with the automaton Do, 
from Lemma [8] In more details, the set of vertices V’ of G’ consists of vertices 
of the form v’ = (v,q), where v is a vertex in G and q is a state in Dor. The 
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owner of the vertex (v,q) is the owner of the vertex v, and the initial vertex is 
(t, qo), where + is the initial vertex in G and qo is the initial state in De, z. We 
have the edge e’ = (v,q) — (v',q') in G’ if e = v > v' is an edge in G with the 
colour z(e) = c, and ô = q & q' is a transition in Do,r. The edge e’ is assigned 
the priority (6) in G’, where 2 is the priority function of the automaton Do, Ff. 
The game g’ then is such that Eve wins G if and only if Eve wins g’. 


Lemma 4. Let G be a Muller game on an arena consisting of m vertices with 
a Muller condition (C,F) whose Zielonka tree Zc r has n leaves and height h. 
Then, G can be converted to an equivalent parity game G’ which has mn many 
vertices and h + 1 priorities. 


2.6 2-dimensional parity game 


Multi-dimensional parity games were introduced by Chatterjee, Henzinger and 
Piterman, where they called it generalised parity games |13]. For our purposes, 
it suffices to consider 2-dimensional (2-D) parity games, which is what we define 
now. 

A 2-dimensional parity game G is similar to a parity game, but we now have 
two priority functions mı : E — [0, dı] and m2 : E — [0,d2] on E. Any infinite 
play in the game is winning for Eve if the following holds: if the play satisfies 
Tı, then it satisfies Tə. 

We say that Adam wins the game otherwise. We call the problem of deciding 
whether Eve wins a 2-D parity game as 2-D PARITY GAME. 


2-D PARITY GAME: Given a 2-D parity game G, does Eve win G? 


If Eve has a strategy to win a 2-D parity game, then Eve has a positional winning 
strategy to do so, i.e. she can win by always choosing the same edge from each 
vertex in V3, which is given by a function ø : V3 — E. This can be inferred 
directly from seeing the 2-D parity game as a Rabin game, which are known to 
have positional strategies for Eve [17]. Furthermore, given a positional strategy o 
for Eve in a 2-D parity game (or a Rabin game), one can check in polynomial time 
if ø is a winning strategy [17]. This gives us a nondeterministic polynomial time 
procedure to decide if Eve wins a given 2-D parity game. In 1988, Emerson and 
Jutla established NP-hardness for Rabin games . This was later extended 
by Chatterjee, Henzinger, and Piterman in 2007 to show NP-hardness for 2-D 
parity games as well [I3]. 


Theorem 5 ({13]). The problem of deciding whether Eve wins a given 2-D 
parity game is NP-complete. 


Remark 6. Chatterjee, Henzinger and Piterman give a slightly different and a 
more natural definition of 2-D parity games [13], where the winning condition for 
Eve requires every play to satisfy either of two given parity conditions. It is easy 
to see, however, that both definitions are log-space inter-reducible to each other, 
by dualising the first parity condition. Our definition, although less natural, 
makes the connection to simulation games and our reductions in Sections 
and [4] more transparent. 
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2.7 Simulation 


We say a parity automaton A simulates another parity automaton B if for any 
(finite or infinite) run on B, there is a corresponding run on A on the same word 
that can be constructed on-the-fly such that if the run in B is accepting, so is the 
corresponding run in A. This is made more formal by the following simulation 
game. 


Definition 7 (Simulation game). Given nondeterministic parity automata 
A = (Q,2,q0, 44, 24) and B = (P, X, po, Ag, 2g), the simulation game be- 
tween A and B, denoted Sim(A,B), is defined as a two player game between 
Adam and Eve as follows, with positions in P x Q. A play of the simulation 
game starts at the position (po, qo), and has w many rounds. For each i € N, the 
(i +1)" round starts at a position (p;,qi) € P x Q, and proceeds as follows: 


— Adam selects a letter a € X, and a transition pi > pi41 in B. 
— Eve selects a transition q; 4 qi+ı on the same letter in A. 


The new position is (pi41,Qi41), for another round of the play. 

The player Eve wins the above play if either her constructed run in A is 
accepting, or Adam’s constructed run in B is rejecting. If Eve has a winning 
strategy in Sim(A, B), then we say that A simulates B, and denote it by BS A. 


We call the problem of checking whether a parity automaton simulates another 
as SIMULATION: 


SIMULATION: Given two parity automata A and B, does A simulate 8? 


The simulation game Sim(A, B) can naturally be seen as a 2-D parity game, 
where the arena is the product of two automata with Adam selecting letters and 
transitions in A and Eve transitions in $, and the priority functions yı and x2 
based on corresponding priorities of transitions in A and B respectively. Since 
2-D PARITY GAME can be solved in NP, SIMULATION can be solved in NP as 
well. 


2.8 History-determinism 


A history-deterministic (HD) parity automaton is a nondeterministic parity au- 
tomaton in which the nondeterminism can be resolved ‘on-the-fly’ just based 
on the prefix read so far, without knowing the rest of the word. The history- 
determinism of a parity automaton can be characterised by the letter game, 
which is a 2-player turn-based game between Adam and Eve, who take alternat- 
ing turns to select a letter and a transition in the automaton (on that letter), 
respectively. After the game ends, the sequence of Adam’s choices of letters is 
an infinite word, and the sequence of Eve’s choices of transitions is a run on that 
word. Eve wins the game if her run is accepting or Adam’s word is rejecting, and 
we say that an automaton is history-deterministic if Eve has a winning strategy 
in the history-determinism game. 
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Definition 8 (Letter game). Given a parity automaton A = (Q, X, qo, A, 2), 
the letter game of A is defined between the two players Adam and Eve as follows, 
with positions in Q x X*. The game starts at (qo,€) and proceeds in w many 
rounds. For each i € N, the (i+1)'" round starts at a position (qi, wi) E€ Q x &", 
and proceeds as follows: 


— Adam selects a letter a; € X 
— Eve selects a transition q; 1 qi+1ı E A 


The new position is (qi+1, Wi+1), where wi41 = Widi. 

Thus, the play of a letter game can be seen as Adam constructing a word 
letter-by-letter, and Eve constructing a run transition-by-transition on the same 
word. Eve wins such a play if the following holds: if Adam’s word is in L(A), 
then Eve’s run is accepting. 


We note that the letter game is an w-regular game: the set of winning plays P for 
Eve are sequences of alternating letters and transitions, so that the word formed 
by just the letters is accepting in A, while the run formed by just the transitions 
is rejecting. Since parity automata can be determinised, it is clear that P is an 
w-regular language, hence the letter game is an w-regular game, and therefore 
the letter game is determined [32122]. 

If Eve has a winning strategy on the letter game of A, then A is said to 
be history-deterministic. We are interested in the problem of checking whether 
a given parity automaton is history-deterministic, which we shall denote by 
HISTORY-DETERMINISTIC. 


HISTORY-DETERMINISTIC: Given a parity automaton A, is A history- 
deterministic? 


2.9 Token games 


Token games, or k-token games are defined on an automaton and are similar to 
letter games. Similar to as in a letter game, Adam constructs a word letter-by- 
letter and Eve constructs a run transition-by-transition on the same word over 
w many rounds. But additionally, Adam also constructs k runs transition-by- 
transition on that word. The winning objective of Eve requires her to construct 
an accepting run if one of k Adam’s runs is accepting. 


Definition 9 (k-token game). Given a nondeterministic parity automaton 
A= (Q, X, ,qo, A, 2), the k-token game of A is defined between the two players 
Adam and Eve as follows, with positions in Qx Q*. The game starts at (qo, (qo)*) 
and proceeds in w many rounds. For each i € N, the (i +1) round starts at a 
position (qi, (p1, p2,- ,pE)) € Q x QF, and proceeds as follows: 


— Adam selects a letter a; € X 

— Eve selects a transition qi “, GtiE A 

— Adam selects k transitions p! => p} 2 li, p2 e pk Ss pE 
Pi — Pipi: Pi — Pi+10' tt Pi > Pipis 
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The new position is (qi+1, (Plp1; Pro. rads from where the (i +2) round 
begins. 

Thus, in a play of the k-token game, Eve constructs a run and Adam k runs, 
all on the same word. Eve wins such a play if the following holds: if one of 
Adam’s k runs is accepting, then Eve’s run is accepting. 


Bagnol and Kuperberg have shown that for any parity automaton A, the 
2-token game of A, and the k-token game of A for any k > 2, are equivalent. 


Lemma 10 ([8]). Given a parity automaton A, Eve wins 2-token game of A if 
and only if Eve wins the k-token game of A for all k > 2. 


If Ais a nondeterministic Biichi or co-Biichi automaton, then Eve wins the 2- 
token game of A if and only if A is history-deterministic , and it is conjectured 
that this result extends to all parity automata. 


TWO-TOKEN CONJECTURE: Given a nondeterministic parity automa- 
ton A, Eve wins the 2-token game of A if and only if A is history- 
deterministic. 


3 Simulation is NP-hard 


In this section, we show that the problem of deciding if a parity automaton 
simulates another is NP-hard, by giving a reduction from the problem of deciding 
whether Eve wins a 2-D parity game, which was shown to be NP-complete by 
Chatterjee, Henzinger and Piterman [I3]. Since a simulation game can be solved 
in NP (see Section 2.7), we obtain NP-completeness. 


Theorem 11. Given two parity automata A and B, deciding if A simulates B 
is NP-complete. 


Since A simulates G if and only if Eve wins the simulation game, which is 
a 2-dimensional parity game (see Section (2.7), and deciding if Eve wins a 2-D 
parity game is in NP [I3], we get that the problem of checking for simulation is 
in NP. Hence, we show that SIMULATION is NP-hard in the rest of this section, 
by giving a reduction from 2-D PARITY GAME. 

Let G be a two-dimensional parity game played on the arena G = (V, E), with 
two priority functions x; and y2. We recall that the winning condition for Eve 
in such a game requires a play to satisfy the y2-parity condition if the .1-parity 
condition is satisfied (see Section [2.6). 


Overview of the reduction. We shall construct two parity automata H and 
D such that H simulates D if and only if Eve wins G. The automata H and D are 
over the alphabet EU {$}, where $ is a letter added for padding. The automaton 
D is deterministic, while the automaton has nondeterminism on the letter $ 
and contains a copy of D. 
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Adam, by his choice of letter in Sim(H,D), captures his moves from Adam 
vertices in G. Similarly, Eve, by means of choosing her transition on $ in H, 
captures her moves from Eve vertices in G. After each $-round in Sim(H,D), 
we require Adam to ‘replay’ Eve’s choice as the next letter. Otherwise, Eve can 
take a transition to the same state as Adam (recall that H contains a copy of 
D), from where she wins the play in Sim(H,D) by copying Adam’s transitions 
in each round from here on-wards. The priorities of D are based on %1, while the 
priorities of H are based on x2. This way D and H roughly accept words that 
correspond to plays in G satisfying x; and x2 respectively. 

We first present our reduction on an example 2-D parity game whose sub- 
game consists of vertices u,v,v’,w,w’ with edges between them as shown in 
Fig. |1| For Adam’s vertex u, we have corresponding states up in D and up in 
H. An Adam move from u in G corresponds to one round of Sim(H,D) from 
the position (up, uz). In G, Adam chooses an outgoing edge, say e = (u, v) from 
u such that xi(u) = cı and x2(u) = c2. This corresponds to Adam choosing 
the letter e in Sim(H,D). We then have the corresponding unique transitions 
up = vg in D and up £2, Yy in H, and hence the simulation game goes to 
(vg,vg). An Eve move from v in G corresponds to two rounds of the simulation 
game from (vg, vy). In Sim(H,D), Adam must select a letter $ and the unique 


$ transition vg 20, vp on D, since $ is the only letter on which there is an 
outgoing transition from vg. Eve must now select a transition on $ from vy. 
Suppose she picks vy 2o, (vH, f) where f = (v, w) is an outgoing edge from v 
in G with xı(f) = cs and y2(f) = ce. This corresponds to Eve selecting the edge 
f from her vertex v in G. The simulation game goes to the position (vp, (vy, f)). 
From here, Adam may select any outgoing edge from v as the letter. If he picks 


f! = (v,w’) and the transition vp 225 wh, then Eve can pick the transition 


(va, f) bance wh and move to the same state as Adam: such transitions are 
indicated by dashed edges in Fig. |1| From here, Eve can win Sim(H,D) by 
simply copying Adam’s transitions. Otherwise, Adam picks the edge f as the 
letter, same as Eve’s ‘choice’ in the previous round, resulting in the transition 
UD pus wp in D and (vy, f) i, wy in H, and the simulation game goes to 
the position (wp, wz), from where the game continues similarly. 


The reduction. We now give formal descriptions of the two parity automata 
D and H such that H simulates D if and only if Eve wins G. We encourage the 
reader to refer to Fig. [I}while reading the construction of the automata described 
below. 

Both automata D and H are over the alphabet X = EU {$}. The automaton 
D is given by D = (P, X, po, Ap, 2p), where the set P consists of the following 
states: 


— states up for each Adam vertex u € W, 
— states vg and vp for each Eve vertex v € V3. 
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Fig. 1. A snippet of a game G, and the corresponding automata D and H constructed 
in the reduction. The Adam vertices are represented by pentagons and Eve vertices by 
squares. The automaton D is deterministic, and H contains a copy of D. 
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The state pp = tp is the initial vertex, where « is the initial vertex of the game 
G. The set Ap consists of the following transitions with their priorities (given 
by 2p) as indicated: 
— transitions up bes vg for every edge e = (u,v) in G such that u € W is 
an Adam-vertex in G, 


TF :0 x š 
— transitions vg ae, up for every v € V3 that is an Eve-vertex in G, 


— transitions vp La, wp for every edge f = (v, w) in G such that v € V3 


is an Eve vertex in G 


The automaton H is given by H = (Q, X,q0, Ay, 2y), where the set Q 
consists of the following states: 


— states uy for each Adam vertex u € YW, 

— states vy for each Eve vertex v € V3, 

— states (vy, f) for each edge f = (v,w) in G such that v € Va is an Eve 
vertex, 

— all states in P, the set of states of D. 


The state go = ty is the initial vertex. The set Ay consists of the following 
transitions with their priorities (given by Ny) as indicated: 
— transitions uy AGN vy for every edge e = (u,v) in G such that u € Ķ is 
an Adam-vertex in G, 
— transitions vH au, (vy, f) for every edge f = (v,w) in G that is outgoing 
from an Eve-vertex v € Va, 


f) f:x2(f) 


— transitions (vq, wy for every edge f = (v, w) in G outgoing from 


an Eve-vertex v € Vy, 

— transitions (vq, f) sa, w'p for every edge f’ = (v, w’) Æ f in G outgoing 
from an Eve-vertex v € Va, 

— all transitions of D. 


Note that, by construction, H contains a copy of D as a sub-automaton. 


Correctness of the reduction. We now show that Eve wins the simulation 
game Sim(H,D) if and only if Eve wins the game G. Call any play of the 
simulation game uncorrupted if the following holds: whenever Eve’s state in H 
is at (vy, f) at the start of a round of Sim(H,D), Adam plays the letter f. 
If Adam plays a letter f’ Æ f, then we call such a move corrupted. Any play 
consisting of a corrupted move is called a corrupted play. 

It is clear that Eve wins any play of Sim(H,D) that is corrupted, since a 
corrupted move causes Eve’s state in H and Adam’s state in D to be the same 
in Sim(H, D). Then, both Eve’s and Adam’s runs are identical and determined 
by the choices of Adam’s letters. In particular, Eve’s run is accepting if Adam’s 
run is. 

Thus, it suffices to consider only uncorrupted plays. We first observe an 
invariant that is preserved throughout any uncorrupted play of Sim(H, D). 
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Invariant: At the start of any round of the simulation game Sim(H, D) 
following an uncorrupted play: 
— Adam’s state is at up for some u € W if and only if Eve’s state is 
at ug 
— Adam’s state is at vg for some v € V3 if and only if Eve’s state is at 
VH 
— Adam’s state is at vp for some v € V3 if and only if Eve’s state is at 
(vy, f) for some edge f that is outgoing from v. 


This invariant is easy to observe from the construction, and can be shown by a 
routine inductive argument. 

Note that if Adam constructs the word w = eof foe1$ fı ...—which we denote 
by (e:$fi)i>0 for succinctness—in an uncorrupted play of Sim(H, D), then Eve’s 
run on H is uniquely determined, since the letter f; indicates how nondetermin- 
ism on H was resolved by Eve on the i” occurrence of $ in Sim(H, D). Thus, 
any uncorrupted play in the simulation can be thought of as Adam selecting 
the e;’s and Eve selecting the f;’s, resulting in the word w = (e;$f;)i>0 being 
constructed in the simulation game. Note that then, by construction, (e;f;)i>0 is 
a play in G. Conversely, if (e; fi)i>0 is a play in G, then there is an uncorrupted 
play of Sim(H,D) whose word is w = (e;$f;)i>0- 

Furthermore, observe that the transitions on a letter e € E in D and H 
in any uncorrupted play have the priorities xı(e) and y2(e) respectively, while 
transitions on $ have priority 0. Thus, in a uncorrupted play of Sim(H, D) whose 
word is (e;$f;)i>0, the highest priorities occurring infinitely often in the run on D 
and H are the same as the highest \1-priority and y2-priority occurring infinitely 
often in the play (e;f;);>0 respectively. 

Thus, an uncorrupted play in Sim(H,D) whose word is w = (e;$f;)i>0 is 
winning for Eve if and only if the play (e;f;);>0 in G is winning for Eve. Since 
Eve wins any corrupted play, the equivalence of the games G and Sim(H,D) 
follows easily now. If Eve has a winning strategy in G, she can use her strategy 
to select transitions so that the word w = (e;$f;);>0 that is constructed in any 
uncorrupted play p of Sim(H,D) corresponds to a winning play for her in G, 
and hence p is winning in Sim(H,D). If Adam ever makes a corrupted move, 
she wins trivially. 

Conversely, if she has a winning strategy in Sim(H,D), then she can use her 
strategy to choose moves in G so that the play (e; fi)i>0 corresponds to a winning 
uncorrupted play of Sim(H, D) in which the word (e;$f;);>0 is constructed, thus 
resulting in the play (e;f;);>0 to also be winning for Eve. 


4 Checking History-Determinism is NP-hard 


In this section, we show that the problem of deciding whether a given nondeter- 
ministic parity automaton is history-deterministic is NP-hard, as is the problem 
of deciding whether Eve wins the 1-token game or the 2-token game of a given 
parity automaton. To show this, we reduce from deciding whether Eve wins a 
2-D parity game with priority functions xı and y2 that satisfies the following 
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property: any play satisfying the y2-parity condition also satisfies the .1-parity 
condition. We call such games ‘good 2-D parity games’. We first show in Sec- 
tion [4.1] that deciding whether Eve wins a good 2-D parity game is NP-hard, 
and then use this to show NP-hardness for the problems mentioned above in 
Section 


4.1 Good 2-D parity games 


Definition 12 (Good 2-D parity game). A 2-D parity game G with the 
priority functions x, and x2 is called good if any play in G that satisfies X2 also 
satisfies x1. 


We call the problem of deciding whether Eve wins a 2-D parity game as GOOD 
2-D PARITY GAME. Chatterjee, Henzinger and Piterman’s reduction from SAT 
to 2-D PARITY GAME [13] can also be seen as a reduction to GOOD 2-D PARITY 
GAME, as we show below. 


Lemma 13. Deciding whether Eve wins a good 2-D parity game is NP-hard. 


Proof. We reduce from the problem of SAT. Let ¢ be a Boolean formula over 
the variables X = {x1,%2,--- ,£&m} that is a conjunction of terms t; for each 
i € [1, N], where each term t; is a finite disjunction of literals—elements of the set 
L = {£1, £2, £M, 701, T2," , Uy}. We shall construct a good 2-D parity 
game Gg such that Eve wins Gg if and only if ¢ has a satisfying assignment. 

Let T = {ti,to,--- ,tw} be the set of all terms in ¢. The game Gy has the 
set TU L as its set of vertices. The elements of L are Adam vertices, while the 
elements of T are Eve vertices. We set the element x; in L to be the initial 
vertex. Each Adam vertex l in L has an outgoing edge e = (l, t) to every term t 
in T, and every Eve vertex t € T has an outgoing edge f = (t,l) to a literal l if 
l is a literal in t. Thus, each play in the game Gg can be seen as Adam and Eve 
choosing a term and a literal in that term in alternation respectively. 

The game Gy has priority functions xı and x2. To every edge e = (l, t) that 
is outgoing from an Adam vertex, both priority functions xı and y2 assign e the 
priority 0, i.e., x1(e) = x2(e) = 0. Every edge e = (t,l) that is outgoing from an 
Eve vertex is assigned priorities as follows: 


2j+2 ifl=a; 2j if l = zj 
xe) = 43. : 7 wale) =A. y 
2j+1 ifl=->zj 2j +1 ifl=-a2; 


This concludes our description of the game Gg. We now show that Gy is a 
good 2-D parity game, which Eve wins if and only if ¢ is satisfiable. 


Gye is a good 2-D parity game. Let p be a play in Gg that satisfies the x2 
parity condition. If 2c is the largest x2-priority occurring infinitely often in p, 
then by construction, 2c + 2 is the largest .1-priority occurring infinitely often 
in the p, which is also even. Thus, p satisfies the x; parity condition. 
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If ¢ is satisfiable, then Eve wins Gy. Let f : {x1, £2, æm} > {T, L} 
be a satisfying assignment of ¢. Let ø be a function which assigns, to each term 
ti, a literal l € t; that is assigned T in f. Consider the Eve-strategy oa in Gg 
defined by o3(t) = (t,a(1)). We claim that o3 is a winning strategy. Indeed, let 
p be a play in Gy following o3, and consider the largest i such that x; or 72; 
appear infinitely often in p. Since oJ is obtained from a satisfying assignment, we 
know that either only x; appears infinitely often, or only ~x; appears infinitely 
often. In the former case, the highest x2 priority appearing infinitely often is 22, 
which is even, and hence p is winning for Eve. In the latter case, the highest x1 
priority appearing infinitely often is 2i+-1, which is odd, and hence the y1-parity 
condition is not satisfied, implying p is winning for Eve. 


If Eve wins Gg, then ¢ is satisfiable. If Eve wins Gy, then we know she 
can win using a positional strategy since Gy is a 2-dimensional parity game. Let 
oa: T — L be such a strategy, where Eve chooses the edge (t,03(t)) at a vertex 
t. If there are no two terms t,t’ such that o3(t) = x; and o3(t’) = 72; for some 
£i, then consider the assignment ø defined as follows. The assignment o maps 
all variables x that are in the image of ca to T, while any terms x; such that 
neither x; or =z; appear in the image are assigned T and L respectively. It is 
clear then that o is a satisfying assignment, since each term t in ¢ evaluates to 
T. 

Otherwise, if there are terms t,t with oa(t) = z; and o3(t’) = 727;, we claim 
that Adam wins the game Gy. Adam can alternate between picking t and t’, 
and then the highest xı priority appearing infinitely often is 2i + 2 while the 
highest x2 priority appearing infinitely often is 2i+1. This implies that the play 
is winning for Adam, which is a contradiction since g3 is a winning strategy for 
Eve. 


4.2 NP-hardness of checking history-determinism 


We now show that deciding the history-determinism, whether Eve wins the 1- 
token game, and whether Eve wins the 2-token game of a given parity automaton 
is NP-hard (Theorem [15}. Much of the work towards this has already been done 
in the reduction from 2-D PARITY GAME to SIMULATION given in Section [| We 
show that the automaton H that is constructed when using this reduction from 
a good 2-D parity game G is such that Eve wins G if and only if H is history- 
deterministic. Since GOOD 2-D PARITY GAME is NP-hard (Lemma|13), we get 
that HISTORY-DETERMINISTIC is NP-hard as well. 


Lemma 14. Checking whether a given nondeterministic parity automaton is 
history-deterministic is NP-hard. 


Proof. Let us consider a good 2-D parity game G. Recall the construction of the 
automata H and D in Section [| which is such that Eve wins G if and only if H 
simulates D. We will show that if G is a good 2-D parity game, then the following 
statements are equivalent. 
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1. Eve wins G. 
2. H simulates D. 
3. H is history-deterministic. 


The equivalence of 1 and 3 would then conclude the proof. The equivalence of 1 
and 2 has already been shown in the proof of Theorem [L1] and we now focus on 
showing that 2 and 3 are equivalent. 

Towards this, let X = E U {$}, and consider the languages £; over X con- 
sisting of the words (e;$f;);>9 such that (e;f;);>0 is a play in G that satisfies 
Xj, for j = 1,2. By construction, we know L(D) = £1, and L(H) = £1 U Lo. 
Furthermore, since G is good, we know that Lı D £2 and hence L(D) = L(H). 
Observe that by construction, D is deterministic. 

If H is history-deterministic, then since L(D) = L(H), Eve wins the simula- 
tion game between H and D: she can use her strategy in the letter game of H 
to play in Sim(H,D), ignoring Adam’s transitions in D. 

The converse direction follows from [24] Theorem 4.1], where Henzinger 
and Piterman show that if a nondeterministic parity automaton M simulates a 
language-equivalent deterministic parity automaton, then M is history-determin- 
istic. We include a proof nevertheless, for self-containment. Supposing H simu- 
lates D, Eve can use her winning strategy in Sim(H,D) to win the letter game 
of H as follows. Eve, during the letter game of H, will keep in her memory, a play 
of the game Sim(H,D). On each round in the letter game of H, Adam gives a 
letter, and Eve, in the game Sim(H,D), lets Adam pick the same letter and the 
unique transition on that letter in D. She then uses her strategy in Sim(H, D) 
to pick a transition in H, and she plays the same transition in the letter game 
of H. We claim that any resulting play of the letter game of H if Eve plays as 
above is winning for Eve. Indeed, if Adam constructs an accepting word in H, 
then it is accepting in D as well. Hence, since D is deterministic, Adam’s run on 
D in the simulation game between H and D that is stored in Eve’s memory is 
accepting. Since Eve is playing according to a winning strategy in Sim(H,D), 
Eve’s run in H, which is the same in Sim(H,D) and the letter game of H, is 
accepting as well. Hence, Eve wins the letter game of H, and thus H is history- 
deterministic. 


We also argue in the full version of the paper [34] that the automaton #H in 
proof of Lemma|l14]above is such that Eve wins the 1-token game of H if and only 
if Eve wins the 2-token game of H if and only if H is history-deterministic. This 
gives us that checking whether Eve wins the 1-token game or the 2-token game 
of a parity automaton is NP-hard. Since 1-token games can naturally be seen 
as a 2-D parity game, we get that deciding whether Eve wins the 1-token game 
of a given parity automaton is in NP, and hence the problem is NP-complete. 


Theorem 15. The following problems are NP-hard: 


1. Given a parity automaton A, is A history-deterministic? 
2. Given a parity automaton A, does Eve win the 2-token game of A? 


Additionally, the following problem is NP-complete: Given a parity automaton 
A, does Eve win the 1-token game of A? 
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5 Language Containment 


In this section, we consider the following problem: 


HD-AUTOMATON CONTAINMENT: Given two parity automata A and B 
such that B is history-deterministic, is L(A) C L(B)? 


While the problem of checking language inclusion between two non-deterministic 
parity automata is PSPACE-complete (regardless of whether the parity in- 
dex is fixed or not) [B0], the same for deterministic parity automata is NL- 
complete Theorem 1]. For history-deterministic parity automata with fixed 
parity indices, however, the problem of language inclusion reduces to checking 
for simulation (Lemma [16}, which can be solved in polynomial time when the 
parity indices are fixed [13]. This gives us that checking for language inclusion 
between two history-deterministic parity automata with fixed parity index can 
be done in polynomial time (Corollary [17}. This observation has been treated 
as folklore, and we prove it here for completeness. 


Lemma 16 ([37/9]). Given a nondeterministic parity automaton A and a 
history-deterministic parity automaton B, the following are equivalent: 


1. B simulates A 
2. L(A) C L(B) 


Proof. (1) = (2): Fix og to be a winning strategy for Eve in Sim(B, A). Let w 
be a word accepted by A via an accepting run p. Consider a play of Sim(B, A) 
where Adam constructs the run p on the word w, and Eve plays according to 
og. Then, the run in B that Eve constructs must be accepting, and hence w is 
accepted by B. 

(2) = (1): Let og be a winning strategy for Eve in the letter game of B. 
Consider the strategy for Eve in Sim(B,A) where Eve chooses the transitions 
on B according to og, ignoring Adam’s transitions in A. If Adam constructs an 
accepting run in A on a word w in Sim(B, A), then w € L(A) C L(B). Hence 
op would have constructed an accepting run in B in Sim(B, A). It follows that 
Eve wins Sim(B, A), and hence B simulates A. 


Corollary 17. Given a nondeterministic parity automaton A and a history- 
deterministic parity automaton B such that both A and B have priorities in |d 
for a fixed d, the problem of whether L(A) C L(B) can be decided in polynomial 
tame. 


We now focus on the problem HD-AUTOMATON CONTAINMENT when the 
parity index is not fixed. From Lemma [16] we know that this can be reduced to 
SIMULATION. Since SIMULATION is in NP [13], we get an immediate NP-upper 
bound for HD-AUTOMATON CONTAINMENT [37] Lemma 3]. We show that we can 
do better, in quasi-polynomial time, by giving a polynomial time reduction to 
finding the winner in a parity game [027]. 

Towards this, let us fix a nondeterministic parity automaton A and a history- 
deterministic parity automaton B over the alphabet X throughout the rest of 
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this section, for which we want to decide if L(A) C L(B). Suppose that A has 
nı states and priorities in [dı], and B has nə states and priorities in [də]. 

It is well known that every such parity automaton A can be converted effi- 
ciently to a language-equivalent nondeterministic Biichi automaton A’ that has 
at most (nı - dı) states [[4]38]. Then, from Lemma [16] it suffices to check if Eve 
wins the game Sim(B, A’). Note that Sim(B, A’) is a 2-D parity game G with 
(nı + dy < ng -|'|)-many vertices that has the priority functions xı : V — [1,2] 
and x2 : V — [d2], where V is the set of vertices of G. 

The game G can be viewed equivalently as a Muller game with the condition 
(C, F), where C = [1, 2] x [dz] and F consists of sets F C C such that if max(F |1) 
is even, then max(F|2) is even. Here, F|; for i € {1,2} indicates the projection 
of F onto the it” component. Call the Zielonka tree (Definition [2) of this Muller 
condition as Zq,. We shall show that the size of Za, is polynomial in də. 


Lemma 18. The Zielonka tree Za, has ({#) many leaves and its height is də. 


The proof of the lemma, obtained via an inductive argument, can be found 
in the full version of the paper [84]. Lemma [18] allows us to use Lemma f on 
Sim(B, A’) to obtain an equivalent Parity game G’ with (n+ d+ n2- |X| -|F]) 
vertices which has də + 1 priorities, such that Eve wins Sim(B, A’) if and only 
if Eve wins G’. 


Lemma 19. Given a nondeterministic parity automaton A with nı states and 
a history-deterministic parity automaton B with ng states whose priorities are 
in [dg] that are both over the alphabet X, the problem of deciding whether L(A) 
is contained in L(B) can be reduced in polynomial time to finding the winner of 
a parity game G which has (nı - dy + n2-|Z|-[¥%]) many vertices and dz + 1 
priorities. 


Since parity games can be solved in quasi-polynomial time[I1]27], Lemma 
implies that the problem of language containment in a history-deterministic 
automaton can be solved in quasi-polynomial time as well. 


Theorem 20. Given a nondeterministic parity automaton A with nı states and 
priorities in |dı], and a history-deterministic parity automaton B with no states 
whose priorities are in [dz], checking whether the language of A is contained in 
the language of B can be done in time 


(ny - dy © no- d2- Leyes), 


6 Discussion 


We have shown NP-hardness for the problem of checking for simulation be- 
tween two parity automata (when their parity indices are not fixed). We have 
also established upper and lower bounds of several decision problems relating 
to history-deterministic parity automata. The most significant amongst these, 
in our view, is the NP-hardness for the problem of deciding if a given parity 
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automaton is history-deterministic, which is an improvement from the previous 
lower bound of solving a parity game [28]. 

There still remains a significant gap between the lower bound of NP-hardness 
and the upper bound of EXPTIME for checking history-determinism, however. 
Furthermore, note that even if one shows the two-token conjecture [BH], this 
would only imply a PSPACE-upper bound (when the parity index is not fixed), 
since 2-token games can be seen as Emerson-Lei games |25|. Thus, a natural 
direction for future research is to try to show that the problem of checking for 
history-determinism is PSPACE-hard. 

On the other hand, however, it is also plausible that checking whether Eve 
wins the 2-token game of a given parity automaton can be done in NP. A proof 
for this might show that if Eve wins a 2-token game, then she has a strategy 
that can be represented and verified polynomially. Such an approach, which 
would involve understanding the strategies for the players in the 2-token games 
better, could also yield crucial insights for proving or disproving the two-token 
conjecture (see Section |2.9). 

Boker and Lehtinen showed in their recent survey that for a ‘natural’ class of 
automata T, checking history-determinism for T-automata is at least as hard as 
solving T-games [8]. Interestingly, the problem of checking history-determinism 
over T-automata also has the matching upper bound of solving T-games for 
all classes of automata T over finite words, and over infinite words with safety 
and reachability objectives on which the notion of history-determinism has been 
studied so far [7[21J35]9/20]. Our result of the problem of checking history- 
determinism being NP-hard for parity automata deviates from this trend (unless 
parity games are NP-hard, which would have the drastic and unlikely conse- 
quence of NP = NP N coNP), and demonstrates the additional intricacy that 
parity conditions bring. 
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Abstract. Tight automata are useful in providing the shortest coun- 
terexample in LTL model checking and also in constructing a maximally 
satisfying strategy in LTL strategy synthesis. There exists a translation of 
LTL formulas to tight Biichi automata and several translations of Biichi 
automata to equivalent tight Biichi automata. This paper presents an- 
other translation of Biichi automata to equivalent tight Biichi automata. 
The translation is designed to produce smaller tight automata and it 
asymptotically improves the best-known upper bound on the size of a 
tight Biichi automaton equivalent to a given Biichi automaton. We also 
provide a lower bound, which is more precise than the previously known 
one. Further, we show that automata reduction methods based on quo- 
tienting preserve tightness. Our translation was implemented in a tool 
called Tightener. Experimental evaluation shows that Tightener usually 
produces smaller tight automata than the translation from LTL to tight 
automata known as CGH. 


1 Introduction 


When a model checking algorithm decides that a given system violates a given 
specification, a counterexample showing the undesired system behavior is pro- 
duced. If the system has only finitely many states and it violates the specification 
given by a formula of Linear Temporal Logic (LTL) or directly by a Biichi au- 
tomaton accepting all erroneous behaviors, there exists a counterexample of the 
form u.v” called lasso-shaped or ultimately periodic. A serious research effort 
has been devoted to algorithms that produce short counterexamples, where the 
length of a counterexample u.v” is given by |wv| [7, 12,13, 15, 19, 22, 24]. 

In 2005, Schuppan and Biere [24] defined tight Biichi automata, where each 
lasso-shaped word accepted by such an automaton is accepted by a lasso-shaped 
run of the same length. Hence, the product of a tight automaton A with an arbi- 
trary transition system accepts the shortest lasso-shaped behavior of the system 
that is in the language of A by the shortest lasso-shaped accepting run. This 
property makes tight automata very useful for automata-based model checking 
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algorithms looking for shortest counterexamples, which was the original motiva- 
tion for the definition. Tight automata found another application in autonomous 
robot action planning, where they are used in the algorithm synthesizing a max- 
imally satisfying discrete control strategy while taking into account that the 
robot’s action executions may fail [27]. 


There exist only few algorithms producing tight automata. The oldest is the 
translation of LTL formulas into generalized Biichi automata introduced by 
Clarke, Grumberg, and Hamaguchi [6] in 1994. The fact that this translation cre- 
ates tight automata was shown about 10 years later by Schuppan and Biere [24], 
who named the translation CGH. They extended the translation to handle also 
past LTL operators and implemented it. The implementation produces automata 
in symbolic representation suitable for the model checker NuSMV [5]. 


There are also two constructions transforming Biichi automata into tight 
Biichi automata. The first was introduced by Schuppan [23] and it accepts even 
generalized Biichi automata as input. For a (non-generalized) Biichi automaton 
with n states, this construction creates a tight automaton with O((V2n)?") 
states. The second (and completely different) construction was introduced by 
Ehlers [13] and it produces tight automata of size 200°) states. Kupferman 
and Vardi [20] provided the lower bound 2% as a side result when analyzing 
counterexamples of safety properties. We are not aware of any implementation 
of these constructions. 


This paper presents another construction transforming Büchi automata to 
tight Biichi automata. More precisely, our construction accepts (state-based) 
Biichi automata (BA) or transition-based Büchi automata (TBA) and produces 
tight BA or tight TBA. The construction is similar to the one of Schuppan [23], 
but it produces less states: while Schuppan’s construction creates states that 
represent a sequence of up to 2n states of the original automata, our construc- 
tion creates states representing at most n states of the original automaton and 
these n states are pairwise different (potentially with a single exception). The 
construction gives us an upper bound in O(n! - n) which is strictly below both 
O((V2n)2”) and 20°), We also provide a lower bound in Q(45+!) for any 
transformation of BA into equivalent tight BA or TBA and a lower bound in 
Q((n — 1)!) for any transformation of TBA into equivalent tight BA or TBA. 
Note that the lower bound (2(45+!) is strictly above the previous lower bound 
2°"), Additionally, we show that tight automata can be reduced by quotienting 
with use of an arbitrary good-for-quotienting (GFQ) relation [8] and the resulting 
automaton is equivalent and tight. 


Our paper also delivers some practical results. The tightening algorithm has 
been implemented in a tool called Tightener. The tool can be easily combined 
with other automata tools as it accepts and produces automata in the HOA 
format [2]. Furthermore, it also accepts LTL formulas on input. When Tightener 
receives an LTL formula, it calls the LTL to TBA translation of Spot [10] as the 
first step. We compare Tightener against the CGH translation as this is (as far as 
we know) the only other implemented algorithm producing tight automata. Our 
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experimental evaluation shows that tight automata produced by CGH usually 
have more states than the ones by Tightener. 


Contributions of the paper. The paper brings the following contributions: 


e a construction transforming BA/TBA into tight BA/TBA with the lowest 
theoretical upper bound on the rise of the state space so far, 

e lower bounds on any transformation of BA or TBA into equivalent tight 
BA/TBA that are currently the highest lower bounds, 

e a proof that the automata reduction based on quotienting preserves tight- 
ness, 

e a tool Tightener producing tight BA/TBA from LTL formulas or BA/TBA, 

e an experimental comparison of tight automata by Tightener and CGH. 


Structure of the paper. The following section introduces the basic terminology 
used in the paper. Section 3 formulates some observations crucial for our tighten- 
ing construction, which is then presented in Section 4 together with the implied 
upper bound. Section 5 shows the lower bounds on the tightening process. The 
postprocessing of tight automata is discussed in Section 6. Section 7 describes 
the implementation of our tightening construction in Tightener and Section 8 
compares it to the CGH translation in terms of the sizes of produced tight au- 
tomata. Finally, Section 9 concludes the paper. 


2 Preliminaries 


A transition-based Biichi automaton (TBA) is a tuple A = (Q, X, 6, I, dr), where 


e Q is a finite set of states, 

e X is a finite alphabet, 

e 6CQx x Qisa transition relation, 
ICQ isa set of initial states, and 

ôr C ô is a set of accepting transitions. 


A run of A over an infinite word u = ugu;... € X® is an infinite sequence 
p = (do, Uo; 41) (G1, U1, G2)... E 6” of consecutive transitions starting in an initial 
state go € I. By pi, we denote the transition (qi, Ui, qi+1) from p. A run p is 
accepting if (qi, Ui, qi+1) € Or holds for infinitely many i. An automaton accepts 
a word u if there exists an accepting run over this word. A language of automaton 
A is the set L(A) of all words in ©” accepted by A. Automata A, B are equivalent 
if L(A) = L(B). 

A transition (p, a, q) € 6 is also denoted as p & q. In graphical representation, 
accepting transitions are these marked with the blue dot @. In the following, 
word without any adjective refers to an infinite word. A path in A from a state 
qo to a state qn over a finite word r = rori ...Tn—-1 E X”™ is a finite sequence 
o = (qo, r0, 91) (1,11; 92) --- (Gn—1;n-1; In) € 6” of consecutive transitions. We 
refer to a first state qo of a path as to initial state of the path. We naturally 
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extend the notation for transitions and write that the path ø has the form 
qo > qn. If such a path exists, we say that qn is reachable from qo over r. For 
a word or a run u = uou ..., by ui., we denote its suffix ujui41... and by ui j, 
for i < j, we denote its subpart ujus41...Uj—-1- 

The paper intensively works with lasso-shaped words and runs, which are 
sequences of the form s.l“, where s is called a stem and | Æ € is called a loop. 
Further, s is a minimal stem and l is a minimal loop of a lasso-shaped sequence 
u = s.l” if for each s’,l! satisfying u = s’.l/” it holds |s| + |l] < |s’| + I]. 


Lemma 1. For each lasso-shaped sequence, there exist a unique minimal stem 
and a unique minimal loop. 


Proof. The existence of some minimal stem and loop for each lasso-shaped se- 
quence u is obvious. We prove its uniqueness by contradiction. Assume that 
there are two different pairs s,l and s’,l’ of minimal stem and loop, which 
implies that u = s.l® = s’l/” and |s| + |i] = |s’| + V|. Without loss of gen- 
erality, assume that |s| < |s’| and [| > |V]. As |s| + [J] = |s| + l|, we get 
IY = ws 4y.. = Usr|4r|.. = UY and thus s.l” = s.l/”. However, this is a contra- 
diction with the minimality of s,/ and s’,l’ as |s| + |l’| < |s| +1] = |s| +l]. 


The minimal stem and the minimal loop of a lasso-shaped sequence u is 
denoted by minS(u) and minL(u), respectively. Moreover, we set |minSL(u)| = 
[mins (u)| + |minL(u)| and call it the size of u. 

If p is a lasso-shaped run over a word u, then u is a lasso-shaped word such 
that |minS(u)| < |minS(p)| and |minL(u)| < |minL£(p)]. 

A TBA A is tight [24] iff for each lasso-shaped word u € L(A) there exists an 
accepting lasso-shaped run p satisfying |minSL(u)| = |minSL(p)|. We call such 
runs tight. 

A state-based Büchi automaton (BA) is a tuple A = (Q, X,ô,I, F), where 
Q, X,ô,I have the same meaning as in a TBA and F C Q is a set of accepting 
states. The definition of all terms is the same as for TBA with the exception of 
accepting run. A run p = (qo, Uo, 1) (qi, U1, G2)... € 6” is accepting if qi € F 
for infinitely many 7. Note that BA can be seen as a special case of TBA as 
each BA can be easily transformed into an equivalent TBA only by replacing 
its accepting states F with the set of transitions dr leading to these states, i.e., 
ôr = {(p,4,q) Eô | q E F}. 

Finally, a (state-based) generalized Büchi automaton (GBA) is a tuple A = 
(Q, ',6,I, F), where Q, X,ô,I have the same meaning as in a TBA and F = 
{F\,..., Fk} is a finite set of sets F; C Q. The definition of all terms is the same 
as for TBA, except for an accepting run. A run p = (qo, uo, q1) (q1, U1, G2)... € OY 
is accepting if for each F; € F there exist infinitely many 7 satisfying q; € Fj. 


3 Observations 


First of all, we explain why our definition of TBA considers multiple initial 
states. As every TBA can be transformed into an equivalent TBA with a single 
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Fig. 1. TBA with a single initial state (left) and an equivalent tight TBA with two 
initial states (right). 


initial state, some definitions of TBA consider exactly one initial state. However, 
a tight TBA with one initial state would have only a restricted expressive power. 
Indeed, each TBA can be transformed to an equivalent tight TBA with multiple 
initial states (as we show in the following section), but there exist TBA that 
cannot be transformed into equivalent tight TBA with a single initial state. 


Lemma 2. There exists a TBA such that there is no equivalent tight TBA with 
a single initial state. 


Proof. Let A be the TBA in Figure 1 (left). For the sake of contradiction, assume 
that there is a tight TBA B with one initial state qo and equivalent to A. Then 
B must accept a” and bY. Furthermore, since |minSL(a”)| = |minSL(b”)| = 1 
and B is tight, there must exist accepting self-loops over a and b in qo. However, 
B then accepts for instance a.b” ¢ L(A), which is a contradiction. 


As the (un)tightness of an automaton depends purely on lasso-shaped words 
accepted by the automaton and the corresponding accepting runs, we turn our 
attention to these words. We start with the definition of significant positions in 
a lasso-shaped word u as positions 7 where u;,. = minL(u)”. Formally, we set 


Sign(u) = {k,k+0,k + 20,k + 30,...} 


where k = |minS(u)| and o = |minL(u)|. We first prove that for every lasso- 
shaped word u accepted by a TBA, there exists a lasso-shaped accepting run 
over u. 


Lemma 3. Let A be a TBA. For each lasso-shaped word u € L(A) there exists 
a lasso-shaped accepting run over u of the form T.n”, where 


e 7 is a path over minS(u).minL(u)' for some i > 0 and 
e m is a path over minL(u)* for some k > 0. 


Proof. Let u € L(A) be a lasso-shaped word and p = (qo, Uo, q1)(@1, U1, G2) --- be 
an accepting run of A over this word. We focus on states of this run at significant 
positions, i.e., states qk, Qk-+0; Uk-+20,--- Where k = |minS(u)| and o = |minL(u)|. 
The run and its states at significant positions are depicted in Figure 2. Since A 
has finitely many states, there are positions pı, p2 E€ Sign(u) such that pı < po, 
Gp. = pz, and path pp, », contains an accepting transition. We set T = po,p, and 
T = Pp, p> AS pi, p2 are significant positions, T is a path over minS(u).minL(u)’ 
for some i > 0 and 7 is a path over minL(u)* for some k > 0. As qp, = qp, and 
mt contains an accepting transition, 7.7” is an accepting run over u. 
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mins (w) minL(u) minL(u) minL(u) 
uo U1 Uk-1 Uk Uk+o-1 Uk+o Uk+20-1 Uk+20 
—> go —> q — --) — (k > k+o ke 20 —> 


states at significant positions 


Fig. 2. A run over a lasso-shaped word u = uou: ... with states at significant positions 
typeset in red. 


Once we know that each lasso-shaped word u € L(A) has a lasso-shaped 
accepting run, we also know that there exists at least one accepting lasso-shaped 
run p over u with the minimal size |minSL(p)|. We call such runs minimal. For 
example, consider the word b.(abc)” accepted by the automaton in Figure 3. 
The minimal run for this word is 7.7” with the following minimal stem 7 and 
minimal loop 7. 


b a b c a 
T = Po — pı — p2 > p3 > pa —> TO 


b c a b c a b È a 
T = ro > rı > T2 > Ta > T3 > r4 > T5 > r4 —@> re > ro 


Now we formulate and prove Lemma 4, which says that each minimal run 
p has a specific property regarding repetition of states. The property considers 
states of p at the positions at least |minS(u)| and less than |minSL(p)|. The 
property says that there cannot be the same state twice on the considered posi- 
tions from which the same suffix of u is read. It can be illustrated on the minimal 
run 7.7” mentioned above. If we write the states of this run such that the states 
reading the same suffix of u are vertically aligned (see Table 1), the considered 
states in each column are pairwise different. 


Lemma 4. Let A be a TBA and p = (qo, Uo, 11) (M1, U1, G2)... be a minimal run 
over a lasso-shaped word u € L(A). For each |minS(u)| < m < L< |minSL(p)| 
satisfying Uum.. = w., it holds that the states qm and qı are different. 


Qen gie 
T 


Om On Om Ou O 


Fig. 3. An example of a TBA that is not tight. 


240 M. Jankola and J. Strejéek 


Table 1. Illustration of the property formulated in Lemma 4. Unconsidered states are 
struck through and states at significant positions are typeset in red. 


suffices of u: b.(abe)”  (abc)®  be.(abc)®  c.(abc)” 

states of r.r“: pe pi p2 P3 
pa ro Tı 
ro T2 Ta 
T4 r5 T4 
r6 s6 I1 
Ve Va YS 
JA 


Proof. Let A be a TBA and p = (qo, Uo, G1) (G1, U1, G2) --. be a minimal run over 
a lasso-shaped word u € L(A). For the sake of contradiction, assume that there 
are positions |minS(u)| < m < l< |minSL(p)| such that um.. = uz, and qm = qı- 
We will show that there exists another lasso-shaped accepting run p’ over u of 
a smaller size than p. This will give us a contradiction with the minimality of p. 


We start with the case that the path p,,; from qm to q contains an accepting 
transition. The equation um.. = ù.. implies that um.. = ur.. = (tm,1)”. Hence, 
P = po.m-(Pm,t)” is a lasso-shaped accepting run over u. Moreover, the size of 
p is smaller than the size of p as |minSL(p’)| < |po,m| + |Pm a| =! < |minSL(p)). 


Now we solve the case when there is no accepting transition in the path 
Pm,- First, assume that pm. is completely included in the minimal stem of 
p, ie, m < l < |minS(p)|. Then we simply exclude p,,, from the stem and 
get an accepting lasso-shaped run p’ over u, which has a shorter stem than 
p. Second, assume that pm is partly in the minimal stem and partly in the 
minimal loop of p, ie., m < |minS(p)| < l. Let ø = pom.-pi.. be the run 
p without the path pm .. Note that p' is again an accepting run over u as 
Um.. = w... As p is lasso-shaped, we know that p1. = ((1,14)minz(p)|)”- Hence, 
p" = po,m-(Pi+|minL(p)|)” is also lasso-shaped. Moreover, the size of p' is smaller 
than the size of p as |minSL(p’)| < m + |minL(p)| < |minS(p)| + |minL(p)| = 
|minSL(p)|. Finally, assume that pm, is completely included in the minimal loop 
of p, i.e., |minS(p)| < m < l. Then we exclude pm, from the minimal loop of p 
and get an accepting run p’ = po, min (p)|:(Plmins(p)l|,m-Pl,\minSL(p)|)” Of a smaller 
size than p. We need to show that p’ accepts u. The run p’ accepts the word 


uw = Ud |mins (p): (Ulmins(p)|,m U, mins L Uo,m-(Uz \minSL(p)| UminS(p)|,m)” = 
As U|minS(p)|,m = U|minSL(p)|,m+|minL(p)|> WE get ul = Uo,m- UimLinint(o) Fur- 
ther, Um.. = Ul. = Um+|minL(p)|.. implies Um.. = Ul.. = (ty eee) and thus 
ul = Uo,m-Um.. = U. 


The next lemma shows that each minimal run over u can be denoted as a 
lasso-shaped structure build from one path over minS(wu) and at most n paths 
over minL(u), where n is the number of states in the automaton. For example, 
the minimal run 7.2” over b.(abc)” presented above can be also denoted as 
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ToT 112.(73 7475)”, where the paths 7; are defined as follows. 


b a b c 

To = po > Ppi T3 = T2 —> T2 7 T3 > T4 
a b c a b c 

Tı = pı — p2 > p3 > p4 T4 = T4 > T5 > T4 —@> re 
a b ë a b c 

T2 = p4 —> ro > rı > T2 Tg = T6 —> To => Ti = T2 


Note that the stem 797172 is not the minimal stem and 737475 is not the minimal 
loop of the minimal run 7.7”. Further, note that the paths m1,..., T5 start in 
the considered states at significant positions, which are typeset in red and not 
struck through in Table 1. 


Lemma 5. Let A be a TBA with n states and p be a minimal run over a lasso- 
shaped word u € L(A). Then p can be denoted as mom... 71;.(Ti417i42---TR)”, 


where To is a path over minS(u), T1, T2,..., Tk are paths over minL(u), and 
0<i<k<n. Moreover, |minSL(p)| < |\wo7m71...7%| < |minSL(p)| + |minL(w)| 
and the last |no71...7| — |minSL(p)| transitions of nk and t; are identical. 


Proof. Let A be a TBA with n states and p = (qo, Uo, q1)(q1, U1; G2)... be a 
minimal run over a lasso-shaped word u € L(A). The lasso shape of p implies 
that P|minS(p)|... = P|minSL(p)|.. and thus also U|minS(p)|.. = UlminSL(p)|..- This 
means that |minL(p)| = j-|minL(u)| for some j > 0. 

Let i > 0 be the smallest number such that minS(u).minL(u)’ is at least as 
long as minS(p). As |minL(p)| = j -|minL(u)|, then k = i + j is the smallest 
number such that minS(u).minL(u)* is at least as long as minS(p).minL(p). Let 
P1,P2,+++,Pk>Pr+1 be the first k +1 significant positions in u. We set 7 = fo,p, 
to be the prefix of p over minS(u) and, for each 1 < l < k, we set T: = Ppi,piyı 
to be the l-th successive subpart of p over minL(u). The definition of k implies 
that |minSL(p)| < |tom1...7%| < |minSL(p)| + |minL(u)]. 

We have Tonm... ng = minS(p).minL(p).2’, where a’ is a prefix of minL(p) 
such that 0 < |n’| = |wom1...7%| — |minSL(p)| < [Tk]. As |tigatize... Te) = j + 
|minL(u)| = |minL(p)|, we get that tonm: ... m; = minS(p).7’ and this means that 
Tk and r; have the same suffix 7’ of the length |z’| = |7o71... | — |minSL(p)]. 
Note that this holds also in the case when i = 0 because this situation implies 
that 7 = minS(p) and thus 7’ = €. 

As ToT... Ti = minS(p).n' and Toni... Tkg = minS(p).minL(p).x’, we get 
that there exists 7” such that minL(p) = n'.n”. Then 


Tom... Ti (MiMi 42---Tr)” = minS(p).1'.(x" 2)” = mins (p). (T.r). = p. 


It remains to show that k < n. For each significant position p; such that 
1 < l < k, it holds that |minS(u)| < pı < |minSL(p)| and up.. = minL(u)”. 
Lemma 4 says that states of the run p at positions pj, p2,...,pp are pairwise 
different. Hence, k < n. 
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4 Tightening construction and upper bound 


Our tightening construction extends a given automaton A with new states and 
transitions to make it tight. Let n be the number of states in A. Lemmata 3-5 
imply that for each lasso-shaped word u € L(A), there exists an accepting run 
P = Mom... 1i-(Ti417i42---TR)” over u where 0 < i < k < n, To is a path 
over minS(u) and 7,72,...,7% are paths over minL(u). Moreover, the states 
at an arbitrary but fixed position in 71,72,...,7,% are pairwise different with 
the exception of the last x states of 1, for some 0 < x < |minZ(u)|, which are 
identical to the corresponding states in mi. 

To accept a lasso-shaped word u € L(A) by a tight run, the extended au- 
tomaton nondeterministically guesses the moment when minS(u) is read and the 
path To terminates. In this moment, it nondeterministically guesses the num- 
bers i, k and the initial states of 72,...,7, and sets the initial state of 7, to the 
current state of the original automaton. When reading minL(u), it simultane- 
ously tracks these paths and if there are more than one possible successors in 
a path, it chooses one nondeterministically. The extended automaton closes a 
cycle over minL(u) via an accepting transition if the tracked paths 7, 72,...,7% 
form together a path 7,72...7, leading to the first state of 7;,, and such that 
Ti+1Ti+2 -Tk contains at least one accepting transition. 

Note that our tightening construction considers only the cases when k > 2. 
If k = 1, then p can be denoted as 79.7 where 7 is a path over minS(u) and 
Tı is a path over minL(u). This means that the run p of A is tight and we do 
not have to extend the automaton because of the corresponding word u. 


Let A be a TBA with n states. The tightening construction adds to A so- 
called macrostates. Each macrostate s1...5; [Sita---SK]F represents 


e the current states 51, 59,...,8, of paths 71, 79,...,7, where2<k<n, 

e the number 0 < i < k marking the beginning of the loop m;417;42...7, 

e the number? < j < k such that 7; is the leftmost path in this loop containing 
an accepting transition, and 

e the information x € {o,e} whether the accepting transition of 7; has been 
already passed (e) or not (o). 


As the paths 7, 72,...,7 are tracked in a parallel and synchronous way, the 
states $1, 52,...,8, of a macrostate have to be pairwise different with a possible 
exception of states s; = sg. Formally, we define the set of macrostates built from 
the set of states Q as 


Me = { 81...8:[Si¢1---SK] 7 | 2<k< IQI, 0O<i< j< k, xe {0,0}, 
$1,---,8~ E Q where sm = sı implies m = l or m,l € {i, k}}. 


Now we are ready to define a tight automaton At equivalent to A. 


Definition 1. Let A = (Q, X,6,I,ôF) be a TBA. We define the TBA At as 
At = (Q U Mg, X, U ð, IU T ôf U Om), where 
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e ð = 6, U ð Ud3 consists of three kinds of transitions, 
e = {51..-Silsi+1---5%]3 € Mo | sı E€ I, si sk}, and 
e Op = 63. 


The transitions in 6; U dg U 63 involve macrostates. They are defined as follows. 
ôi = {q 4 $1 ---84[8441---8e]7 | qd 4 S1 E ô, Si Æ Sk} 


These transitions are used to nondeterministically choose the numbers i, j,k and 
the initial states of 72,73,...,1, when reading the last symbol of minS(u). If 
minS(u) = £, the nondeterministic choice is done by starting the computation 
in a macrostate of I’. 


62 = { $1---35[Si41---Sk] 5 rie rilri+i rel} | x, x € {0,0}, 
VI<Il<k.ifi<l <j then sı Sr € ôx 6p else sı 4 rı € 6, 
Si = 8% implies ri = Tx, if 8; 4+ rj E Op then x =e else x = *} 


These transitions simultaneously track the progress on the paths T1, T2,...,Tk 
including the information whether 7; has already passed an accepting transition 
or not. The condition sı 4 rı g Or fori <1 <j enforces that 7; is the leftmost 
path on the loop TiTa... Tk containing an accepting transition. 


63 = { 81...8i[Si41---S4]% 4 PP Piga Tels | * € {0,0}, Sk = Ti+1 € Ô, 
Y1I<l<k.ifi<l< j then sı % m4, EN ôr else sı S 141 € Ô, 
Ti ÉTrk, x=% or(j<k ^A 8; 5 rj41 € OF) or (j=k ^ sk Tigi € OF)} 


These accepting transitions can enclose a cycle on macrostates if the last state of 
mı matches the first state of mi41 for each 1 < l < k, the last state of nk matches 
the first state of Ti+ı, and 7; has passed an accepting transition in the past or 
during this step. 


Theorem 1. Let A = (Q, X,ô,I,ôF) be a TBA. Then L(A) = L(A‘). 


Proof. The inclusion L(A) C L(A’) is trivial as each accepting run of A is also 
an accepting run of At. 

We show that L(A‘) C L(A). Let o be an accepting run of At that in- 
volves some macrostates. Note that all macrostates in the run have to use the 
same numbers i, j, k. We construct an accepting run p of A over the same word 
as o. Intuitively, p will consistently use the transitions of some element of the 
macrostates in o, starting with the first element. Each time o uses a transition 
of 63, p will switch to the next element and after the k-th element, it will switch 
back to the (¢ + 1)-st element. 

First we define an auxiliary function g that determines for each | > 0 the 
element of the macrostate in ø that will be followed by the transition pı. 


g(l) if o, € 53 
g(0) =1 gl+1)= <4 g() +1 if o € 63 and g(l) < k 
i+ 1 if o; € 63 and g(l) =k 
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Now we construct p as follows. 


Ol if a, € ô 
p=tq> si if oy =q S 81...5;[8i41.--84]? € 51 
S4(l) 4 ¥ g(14+1) if oy = 81...84[Si+1---Sk]j 4 ririri re] E€ 62 U 63 


One can easily check that p is a run of A over the same word as ø. Further, 
because ø is accepting, it contains infinitely many transitions of 63. Hence, there 
are infinitely many pairs m,/ such that 0 < m < l and 


g(m—1) Aj =g(m) =g(m+1)=...= 9-1) Fg). 


The definition of g implies that om 1, € 63.65.63, which means that the j-th 
element of some macrostate in om, takes an accepting transition in dr. The 
construction of p guarantees that pm, contains the same transition in dr. Hence, 
p contains infinitely many accepting transitions and it is therefore accepting. 


Theorem 2. Let A =(Q,¥,6,1,6r) be a TBA. Then At is tight. 


Proof. Lemma 3 implies that for each lasso-shaped word u € L(A), there exists 
a minimal run of A over u. The validity of the statement then follows directly 
from the properties of minimal runs proven in Lemmata 4 and 5 and from the 
design of the tightening construction. 


4.1 State-based tight automata 


While our tightening construction produces automata with transition-based ac- 
ceptance, the previous tightening constructions [13, 23,24] produce automata 
with state-based acceptance. Some applications [27] also work with tight state- 
based automata on the input. Therefore, we present a transformation of a tight 
TBA to an equivalent BA preserving tightness. 


Let A = (Q,,6,I,6r) be a tight TBA. An equivalent tight BA B can be 
constructed as follows. We define the set of accepting states as duplicates of 
states q E€ Q that have some accepting transition starting in q, i.e., F = {q | 
q>ped rF}. We extend the initial states and the transition relation in such a 
way that whenever the original automaton can use an accepting transition from 
a state q, the resulting state-based automaton can reach the corresponding state 


q and use an analogous transition from it. Formally, the tight BA B equivalent 
to A is constructed as B = (QU F, X, U ', TUT’, F), where 


el ={G|qeI}NF and 
oS ={p>7|p >qEð TEF}UP >g] p> qE br, DIE FU 
a) a me 
U{p>q|p>q€ ðr, DEF}. 
Each accepting run o of B can be transformed to an accepting run p of A 


over the same word simply by replacing each state q € F by the corresponding 
state q. Thus we get L(B) C L(A). 
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Further, each accepting run p of A can be transformed into an accepting 
run g of B over the same word simply by replacing each state q from which 
an accepting transition is taken with the corresponding state g. This implies 
L(A) C L(B). Moreover, when we apply this transformation to a tight run p of 
A, we obtain a tight run o of B. To sum up, the automata A and B are equivalent 
and if A is tight, then B is also tight. 


4.2 Upper bound for tightening 


Lemma 6. Let A be a TBA with n states. The number of states in A‘ is at 
most 


n+2- 2 us H ) O(n! - n3). 


Proof. Let Q be the set of states of A. First we bound the number of macrostates 


of the form 81...5i[Si41---Sk]} € Mg for a fixed i, j,k. There are TW - 2 cases 
where all states s1, 52,..., Sg are pairwise different and TEDT -2 cases where 
S1, S2;,.-.,Sķ—1 are pairwise different and są = s;. The factor 2 comes from 
x € {o,e}. Altogether, Mg contains at most 4 - nom macrostates for fixed 
i,j,k. Further, for a fixed k > 2, there are EED possible pairs of values of 
i,j satisfying 0 < i < j < k. Altogether, the number of macrostates in Mg can 
! k-(k+1 lLk (k+1 
be bounded by yo 4: giy “SY = 2- Dp SE. When we add the 
number n = |Q| of the original states, we get the statement. 


Recall that each BA can be seen as a special case of a TBA. Further, note 
that the transformation of tight TBA to tight BA presented in Section 4.1 only 
doubles the state space in the worst case. Hence, we also proved that each BA or 
TBA with n states can be transformed into an equivalent tight BA with at most 
O(n!-n3) states. This upper bound is tighter (i.e., asymptotically smaller) than 
the upper bound 2°”) by Ehlers [13] and than the upper bound O((V2n)?”) 
derived from the Schuppan’s construction [23] as 


5 Lower bound for tightening 


We present a lower bounds for any transformation of a TBA or a BA to an 
equivalent tight TBA or BA. 


Lemma 7. For eachn > 0, there isa TBA A with n+1 states and an equivalent 
BA A with 2n+1 states such that for every equivalent tight TBA B with the set 


of states Q it holds that 
= n! 
> X — 
Q| 2 (n—k)! 


k=1 
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Fig. 5. The automaton A for n = 2. The construction considers 4 sequences and each 
sequence induced transitions that accept the following words: [s] relates to the word 
ao.b6, [rs] to a1.b7, [r] to a2.b3, and [sr] to a3.b3. 


Proof. Let us fix some n > 0. We construct the TBA A with n+1 states gradually 
as follows. The automaton uses states {qo} U Q’ where qo is the only initial 
state and Q’ contains another n states. The construction works with nonempty 
sequences [s1...5%] of pairwise different states from Q’. For each [s1...s,], we add 
fresh symbols a, b to the alphabet of A and the transitions depicted in Figure 4 to 
the transition relation of A. The automaton accepts a.b” with these transitions. 
The constructed automaton for n = 2 is in Figure 5. The equivalent BA A’ is 
constructed from A by the transformation given in Section 4.1. 

Now we assume that B = (Q, X,ô,I,ôr) is a tight TBA equivalent to A. 
Each [s1...8%] induces the acceptance of a new word a.b” € L(A) = L(B). As 
B is tight and |minSL(a.b”)| = 2, there have to be transitions p “> q € 6 and 


q 2 q € or for some states p € I and q € Q. We prove by contradiction that the 
state q has to be different for each [s1...sx]. 

Let us assume that [s;...5,] and [r1...rx/] are different sequences inducing 
the acceptance of a.b” and a’.b/, respectively, and B accepts these two words 
using transitions p “> q, p’ ay q € 6 and q 4 a4 Bs q € ôr. The situation is 
depicted in Figure 6. We distinguish two cases. 


1. {s1,..., Sk} A {r1,.--, ræ}: Without loss of generality, we assume that 
rj Æ {S1,---, Sk}. As B accepts all words in {a’}.{b, b’}”, it also accepts the 
word a’.b'7~!.b”. However, this word is not in L(A) as A is deterministic and 
after reading a’.b'J~1 it gets to state r; which has no transition over b since 
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b N ty | b' , 
ON e 


Fig. 6. Illustration of the assumption. States p and p’ does not have to be different. 


since rj ¢ {s1,..., Sk}. Hence, L(B) # L(A) and this is a contradiction with 
our assumptions. 
2. {81,...,5~} = {r1,..., rx}: As states in each sequence are not repeating, 
we get k = k’. We use the fact that the only accepting transitions over b and 
b' in A are those from sı and r1, respectively. We distinguish two subcases: 
(a) sı = rı: Let j be the smallest number such that sj Æ rj. As the sets 
of states are equal, there exist j < m,m’ < k, such that sm =r; and 
Tm’ = sj. Consider the run 7.7” of A, where 


a b b b 
T = qo > sı —®> s2 >... sj and 
d J 


b b b b 
T= Sj > Sj41--- > (Sm = rj) > Tj+1 ae > (Tm! = sj). 


As 7m contains no accepting transition, the run is not accepting. Since A 

is deterministic, it is the only run of A over a.b/—!.(b™-Jp'™ J)”, As 

the word is accepted by B, we get a contradiction with L(A) = L(B). 
(b) sı Æ rı: Since the sequences contain the same states, there are some 


Ee b! : 
1 < m,l < k such that sı —> rı and rı — sı. Consider the run 7.7” 
of A, where 
b ™ b! 
T = qo > sı and T = S1 — rı > sı. 
The run is not accepting as the only accepting transitions over b or b' 
starting in sı and rı, respectively, are never taken. Since A is determin- 


istic, it is the only run of A over a.(b/b')”. As the word is accepted by 
B, we get a contradiction with L(A) = L(B). 


To sum up, we proved that every tight TBA B satisfying L(B) = L(A) must have 
at least one state for every nonempty sequence [s1...8%]. This directly implies 


that the number of its states is at least X; TR 


The previous lemma says that for each n > 2, there exists a TBA with n 
states such that the smallest equivalent tight TBA (and thus also the smallest 


equivalent tight BA) has at least ae ct states. This function is clearly 
in 2((n — 1)!) as 


: k=1 (n—1—k)! : 1 | 
aa eae ge 
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Note that the difference between the upper bound O(n! - n?) = O((n — 1)! - n*) 
given in Lemma 6 and the lower bound 2((n — 1)!) is only the factor n4. 
Analogous arguments lead to the statement that for each odd n > 3 there 
exists a BA with n states such that the smallest equivalent tight TBA (and thus 
also the smallest equivalent tight BA) has at least 2(45+!) states. This lower 


bound is above the previously known lower bound 2°) as for each c it holds 
JEN 


6 Postprocessing of tight automata 


This section shows that a standard automata reduction technique called quoti- 
enting [8] preserves tightness. Hence, it can be applied to reduce tight automata 
before they are further processed. 

Consider an automaton with the set of states Q. A preorder C CQ x Qisa 
reflexive and transitive relation. Every preorder defines an induced equivalence 
x = CNJ. Given a state q, we denote by |q] the equivalence class of q with 
respect to a fixed equivalence ~. Furthermore, for every P C Q, by |P] we denote 
the set [P] = {[q] | q € P} of all equivalence classes of states in P. 

Given a TBA A = (Q, X,ô,I, ôr) and a preorder E on Q with its induced 
equivalence ~, the quotient of A is the TBA A/E = ([Q], X, 6’, [I], 04-), where 
5 = {[a] > [p] | q > p € 5} and 5p = {[q] > [p] | q > p € ôr}. 

A preorder E is good for quotienting (GFQ) [8] if L(A) = L(A/C) for each 
TBA A. There exist many preorders that are GFQ, for example various kinds of 
forward or backward simulation or trace inclusion. For their definition and more 
information about automata reduction techniques we refer to the comprehensive 
paper by Clemente and Mayr [8]. 


Lemma 8. Let A be a tight TBA and let © be a GFQ preorder. The automaton 
A/C is tight and L(A) = L(A/E). 


Proof. The language equivalence trivially follows from the definition of GFQ. 
Let us consider an arbitrary lasso-shaped word u € L(A). As A is tight, it has an 


: ins 
accepting run p = 7.7” where 7 has the form qo TW, l and m has the form 
inL 
pa (u) l. The definition of quotient implies that for each accepting run of 
A there exists an accepting run over the same word through the corresponding 
equivalence classes in A/C. Hence, A/E has an accepting run p’ = T'.n'® where 


minS(u) 


T” has the form [qo] ————> [I] and 7’ has the form [I] [l]. It is easy to 
see that |minSL(p’)| < |minSL(p)| and thus p’ is tight. Therefore, the automaton 
A/C is tight. 


minL(u) 


7 Implementation 


We have implemented our tightening construction in a tool called Tightener. 
The tool is written in Python 3.8.15 and it is built upon the library for LTL and 
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w-automata called Spot [10] in version 2.11.4. Spot provides state-of-the-art LTL 
to automata translations, efficient transformations of arbitrary automata in the 
HOA format [2] to equivalent TBA, and some automata reduction techniques, 
in particular direct simulation [8] that is good for quotienting. 

Tightener can take as an input either an LTL formula or an automaton in the 
HOA format. The input is internally translated into an equivalent TBA using 
the functionality provided by the Spot library. The TBA is then transformed 
into a tight TBA or tight state-based BA using the construction presented in 
this paper. The tight automaton is then optionally reduced using Spot’s func- 
tion reduce_direct_sim which performs quotienting by direct simulation. The 
resulting tight automaton is encoded in DOT or in the HOA format. 

Tightener is available in an artifact at Zenodo® and at the project repository* 
under the GNU Public License, version 3 [1]. The tool can be run in the direc- 
tory Tightener_project using the command python Tightener.py [flags] 
"input". The tool supports the following flags. 


-h or --help describes the basic usage of the tool. 

-f or --formula says that the "input" is an LTL formula (e.g., "Fp1 | Fp2") 
on the command line. Tightener uses the same syntax for LTL formulas as 
Spot, see https://spot.lre.epita.fr/ioltl.html. 

-F or --file says that the "input" is a path to a text file containing an LTL 
formula in the format mentioned above. 

-a or --HOA says that the "input" is a path to a file containing an automaton 
in the HOA format. 

-s or --sbacc asks to produce a state-based tight automaton. The tool pro- 
duces tight TBA by default. 

-r or --reduces applies reductions preserving tightness before the tight au- 
tomaton is returned. These reductions are not applied by default. 

-o or --outputHOA outputs the tight automaton in the HOA format. By default, 
the tool returns a tight automaton in DOT format, which can be easily vi- 
sualized, for example at https://dreampuf .github.io/GraphvizOnline/. 
Note that the DOT format does not support multiple initial states. Hence, if 
the returned automaton has multiple initial states, one of them is marked as 
initial and the others are identified by an auxiliary incoming edge labeled 
with init. 


8 Experimental results 


We compare Tightener against the translation of LTL to state-based generalized 
Büchi automata introduced by Clarke, Grumberg, and Hamaguchi [6] and called 
CGH. Schuppan and Biere [24] proved that the automata produced by CGH 
are tight. As far as we know, this is the only existing implementation besides 
Tightener that produces tight automata. Still, the comparison is not entirely 


3 nttps: //zenodo. org/records/10512677 
f https: //gitlab.com/mjankola/tightener/-/tree/main?ref_type=heads 
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Table 2. We compare the tight TBA and BA produced by Tightener against the 
GBA constructed by CGH. For both datasets, the table shows the number [#] and the 
percentage [%] of cases where the corresponding tool provided a tight automaton with 
fewer states than the other tool. Columns avg. size represent the average number of 
states of the automata constructed by the corresponding tool. Columns TO indicate 
the number of timeouts. Cases where Tightener timed out are counted in the CGH 
winning columns, but these cases are excluded from the computation of average size. 


642 random formulas 219 formulas from literature 
tool [#] [%] avg. size TO  [+#] [%] avg. size TO 
Tightener (TBA) 482 75.1% 20.03 44 179 81.7% 37.00 28 
CGH (GBA) 149 23.2% 73.9 0 39 17.8% 161.51 0 
Tightener (BA) 381 59.3% 32.54 44 141 64.4% 60.44 28 
CGH (GBA) 243 37.8% 73.9 0 72 32.8% 161.51 0 


fair as Tightener and CGH have different input and different output: Tightener 
can transform any LTL formula or automaton in the HOA format to tight TBA 
or BA, CGH accepts only an LTL formula and produces a tight GBA. BA can 
be seen as a special case of both TBA and GBA, but the opposite does not 
hold. We provided a transformation of tight TBA into equivalent tight BA in 
Section 4.1. Each GBA can be transformed into an equivalent BA (this so- 
called degeneralization process has been recently significantly improved [3]), but 
the transformation increases the number of states and it does not guarantee to 
preserve tightness. We therefore compare the size of tight GBA produced by 
CGH against the size of tight TBA and tight BA produced by Tightener. 


Since CGH produces tight GBA in symbolic representation, we implemented 
a process that enumerates automata states from this symbolic representation and 
uses the SMT solver Z3 [9] to prune unreachable and contradictory states. In the 
end, we count the number of reachable states. This implementation can be also 
found in our repository in script Tightener_project/CGH_implementation. py. 


We compare CGH and Tightener on two sets of LTL formulas. The first 
dataset contains 642 formulas produced by random formulas generator rand_1t1 
of Spot’s. These formulas are stored in file 1t1DataSet_random.txt in our 
repository. The second dataset consists of 219 formulas taken from literature 
[4,11, 14, 16-18, 21, 25, 26]. We obtained these formulas from the tool gen_1t1 of 
Spot and they are stored in file 1t1DataSet_pattern.txt in our repository. 

We ran the experiments on a machine with an AMD Ryzen 7 PRO 4750U 
processor and 32 GB of RAM. We set 15 minutes timeout limit per task with 
no explicit memory limit. 

Each formula has been translated by Tightener to a tight TBA and to a tight 
BA with reduction switched on in both cases, and by CGH to a tight GBA. 
Table 2 summarizes the cummulative results for the two datasets. One can see 
that Tightener constructs smaller automata in substantially more cases than 
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Fig. 7. The comparison of the number of states of the tight automata produced by 
Tightener and CGH for individual LTL formulas of each dataset. In the top row, 
Tightener produces tight TBA. In the bottom row, it produces tight BA. CGH always 
produces GBA. The red crosses display the cases where Tightener reaches a time limit. 


CGH in both considered datasets and with both settings. However, Tightener 
run out of time in some cases. 

The scatter plots in Figure 7 compare the number of states of the tight au- 
tomata constructed by CGH and Tightener for individual LTL formulas in each 
dataset. Since some of the produced automata are rather large, we use logarith- 
mic scale in all of the scatter plots. The graphs clearly show that Tightener often 
produces dramatically smaller tight automata than CGH. 


8.1 Experiments on formulas for robot action planning 


Tumova et al. [27] introduced a technique that generates control strategies for 
a robot planning problem. They represent the strategies as lasso-shaped words, 
where alphabet is a set of locations and possible actions in the respective location. 
Their approach takes advantage of tight BA to obtain the strategies with the 
shortest length of the stem and the loop. 

The paper contains three LTL formulas representing meaningful properties. 
Table 3 compares the sizes of tight BA obtained from Tightener and tight GBA 
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Table 3. Sizes of tight automata constructed from LTL formulas taken from the study 
of Tumova et al. [27]. TO indicates a timeout. 


number of states 


Tightener CGH 


formula (BA) (GBA) 
GF(Ra A grab A F(R2 A drop)) A GFlight_up 38 224 
GF(((R4 A grab) V (Rs A grab)) A F(R2 A drop)) A GFlight_up 43 317 
G(Ri > Aig “Ri U Ro A (Aigo Ri U R3 A 

(Niza “Ri U (Re A drop) A Nizo ™Ri U Rs A 2 TO 


(Nizs “Ri U (Ra A drop) A (Niza 7Ri U R1))))) A GFlight-up 


from CGH on these formulas. For two of the formulas, Tightener constructed 
dramatically smaller automaton than CGH. On the third formula, Tightener 
produced a tight BA with 2 states while CGH ran out of time. 


9 Conclusions 


In this paper, we presented a new approach for converting TBA or BA to tight 
TBA or BA. We proved that the asymptotical rise of the state space is O(n!- n3), 
which is the smallest upper bound so far reached. Further, we proved the highest 
lower bounds on the rise of the state-space of tight automata so far reached, 
making the theoretical construction of tight automata significantly tighter. We 
also showed that the good-for-quotienting simulations can be used to reduce 
automata while preserving tightness. 


Our tool Tightener opens new ways to construct tight automata as it is 
the first tool that can create tight automata from arbitrary automata in the 
HOA format or from LTL formulas. We compared Tightener against the LTL to 
tight automata translation CGH on two datasets of LTL formulas. Experiments 
show that Tightener constructs smaller tight automata in substantially more 
cases. Moreover, we compared the two tools on three formulas for which a tight 
automaton was explicitly desired before. In all three cases, Tightener provided 
a dramatically better result. 
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Abstract. We study automatic synthesis of systems that interact with 
their environment and maintain privacy against an observer to the inter- 
action. The system and the environment interact via sets J and O of input 
and output signals. The input to the synthesis problem contains, in ad- 
dition to a specification, also a list of secrets, a function cost : [UO > N, 
which maps each signal to the cost of hiding it, and a bound b € N on 
the budget that the system may use for hiding of signals. The desired 
output is an (1/O)-transducer 7 and a set H C J UO of signals that 
respects the bound on the budget, thus }7,-4, cost(s) < b, such that for 
every possible interaction of 7, the generated computation satisfies the 
specification, yet an observer from which the signals in H are hidden, 
cannot evaluate the secrets. 

We first show that the complexity of the problem is 2EXPTIME-complete 
for specifications and secrets in LTL, thus it is not harder than synthesis 
with no privacy requirements. We then analyze the complexity of the 
problem more carefully, isolating the two aspects that do not exist in 
traditional synthesis, namely the need to hide the value of the secrets 
and the need to choose the set H. We do this by studying settings in 
which traditional synthesis can be solved in polynomial time — when the 
specification formalism is deterministic automata and when the system 
is closed, and show that each of the two aspects involves an exponential 
blow-up in the complexity. We continue and study bounded synthesis with 
privacy, where the input also includes a bound on the size of the synthe- 
sized transducer, as well as a variant of the problem in which the observer 
has knowledge about the specification, which can be helpful in evaluating 
the secrets. We study the effect of both variants on the different aspects 
of the problem and provide algorithms with a tight complexity. 


1 Introduction 


Synthesis is the automated construction of correct systems from their specifica- 
tions [2]. While synthesized systems are correct, there is no guarantee about their 
quality. Since designers will be willing to give up manual design only after being 
convinced that the automatic process replacing it generates systems of compa- 
rable quality, it is extremely important to develop and study quality measures 
for automatically-synthesized systems. An important quality measure is privacy: 
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making sure that the system and its environment do not reveal information they 
prefer to keep private. Privacy is a vivid research area in Theoretical Computer 
Science. There, the notion of differential privacy is used for formalizing when an 
algorithm maintains privacy. Essentially, an algorithm is differentially private if 
by observing its output, one cannot tell if a particular individual’s information 
is used in the computation [9,11]. Another related notion is obfuscation in sys- 
tem development, where we aim to develop systems whose internal operation is 
hidden [1,15]. Obfuscation has been mainly studied in the context of software, 
where it has exciting connections with cryptography [1,15]. 


In the setting of automated synthesis in formal methods, a very basic notion 
of privacy has been studied by means of synthesis with incomplete information 
[27,21,7]. There, the system should satisfy its specification eventhough it only has 
a partial view of the environment. Lifting differential privacy to formal methods, 
researchers have introduced the temporal logic HyperLTL, which extends LTL 
with explicit trace quantification [8]. Such a quantification can relate computa- 
tions that differ only in non-observable elements, and can be used for specifying 
that computations with the same observable input have the same observable 
output. The synthesis problem of HyperLTL is undecidable, yet is decidable for 
the fragment with a single existential quantifier, which can specify interesting 
properties [13]. In [18], the authors suggested a general framework for automated 
synthesis of privacy-preserving reactive systems. In their framework, the input 
to the synthesis problem includes, in addition to the specification, also secrets. 
During its interaction with the environment, the system may keep private some 
of the assignments to the output signals, and it directs the environment which 
assignments to the input signals it should keep private. Consequently, the satis- 
faction value of the specification and secrets may become unknown. The goal is 
to synthesize a system that satisfies the specification yet keeps the value of the 
secrets unknown. Finally, lifting obfuscation to formal methods, researchers have 
studied the synthesis of obfuscation policies for temporal specifications. In [32], 
an obfuscation mechanism is based on edit functions that alter the output of the 
system, aiming to make it impossible for an observer to distinguish between se- 
cret and non-secret behaviors. In [10], the goal is to synthesize a control function 
that directs the user which actions to disable, so that the observed sequence of 
actions would not disclose a secret behavior. 


In this paper we continue to study privacy-preserving reactive synthesis. As 
in [18], our setting is based on augmenting the specification with secrets whose 
satisfaction value should remain unknown. Unlike [18], the system and the en- 
vironment have complete information about the assignments to the input and 
output signals, and the goal is to hide the secrets from a third party, and to do 
so by hiding the assignment to some of the signals throughout the interaction. 
As an example, consider a system that directs a robot patrolling a warehouse 
storage. Typical specifications for the system require it to direct the robot so 
that it eventually reaches the shelves of requested items, it never runs out of 
energy, etc. An observer to the interaction between the system and the robot 
may infer properties we may want to keep private, like dependencies between 
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customers and shelves visited, locations of battery docking stations, etc. If we 
want to prevent the observer from inferring these properties (a.k.a,. the secrets), 
we have to hide the interaction from it. Different effort should be made in order 
to hide different components of the interaction (alarm sound, content of shelves, 
etc.). Our framework synthesizes a system that realizes the specification without 
the secrets being revealed, subject to restrictions on hiding of signals. As another 
example, consider a scheduler that should grant access to a joint resource. The 
scheduler should maintain mutual exclusion (grants are not given to different 
users simultaneously) and non-starvation (all requests are granted), while hid- 
ing details like waiting time or priority to specific users. In Examples 1 and 2, 
we describe in detail the application of our framework for the synthesis of such a 
scheduler, as well as its application in the synthesis of a robot that paints parts 
of manufactured pieces. The robot should satisfy some requirements about the 
generated pattern of colors while hiding other features of the pattern. 


Formally, we consider a reactive system that interacts with its environments 
via sets J and O of input and output signals. At each moment in time, the 
system reads a truth assignment, generated by the environment, to the signals 
in J, and it generates a truth assignment to the signals in O. The interaction 
between the system and its environment generates a computation. The system 
realizes a specification y if all its computations satisfy y [25]. We introduce 
and study the problem of synthesis with privacy in the presence of an observer. 
Given a specification y, and secrets p1, ...,WŲk over I U O, our goal is to return, 
in addition to a system that realizes the specification y, also a set H C IUO 
of hidden signals, such that the satisfaction value of the secrets v1,...,Wx is 
unknown to an observer that does not know the truth values of the signals in 
H. Thus, secrets are evaluated according to a three-valued semantics. The use of 
secrets enables us to hide behaviors, rather than just signals. ' Obviously, hiding 
all signals guarantees that the satisfaction value of every secret is unknown. 
Hiding of signals, however, is not always possible or involves some cost. We 
formalize this by adding to the setting a function cost : [UO — N, which maps 
each signal to the cost of hiding its value, and a bound b € N on the budget 
that the system may use for hiding of signals. The set H of hidden signals has 
to respect the bound, thus >) „ey cost(s) < b. 


In some cases, it is desirable to hide the truth value of a secret only when 
some condition holds. For example, we may require to hide the content of selves 
only in some sections of the warehouse. We extend our framework to conditional 
secrets: pairs of the form (6,w), where the satisfaction value of the secret w 
should be hidden from the observer only when the trigger 0 holds. In particular, 
when 0 = w, we require to hide the secret only when it holds. For example, we 
may require to hide an unfair scheduling policy only when it is applied. Note 
that a conditional secret (0,w) is not equivalent to a secret 6 > w or 6 > =y, 
and that the synthesized system may violate the trigger, circumventing the need 


1 Hiding of signals is a special case of our framework. Specifically, hiding of a signal p 
can be done with the secrets Fp and F’np. 
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to hide the secret. For example, by synthesizing a fair scheduler, the designer 
circumvents the need to hide an unfair policy. 


We show that synthesis with privacy is 2EXPTIME-complete for specifica- 
tions and secrets in LTL. Essentially, once the set H of hidden signals is deter- 
mined, we can compose an automaton for the specification with automata that 
verify, for each secret, that the assignments to the signals in (I UO) \ H can 
be completed both in a way that satisfies the secret and in a way that does not 
satisfy it. A similar algorithm works for conditional secrets. 

While the complexity of our algorithm is not higher than that of LTL synthe- 
sis with no privacy, it would be misleading to conclude that handling of privacy 
involves no increase in the complexity. The 2EXPTIME complexity follows from 
the need to translate LTL specifications to deterministic automata on infinite 
words. Such a translation involves a doubly-exponential blow-up [22,20], which 
possibly dominates other computational tasks of the algorithm. In particular, 
two aspects of synthesis with privacy that do not exist in usual synthesis are a 
need to go over all possible choices of signals to hide, and a need to go over all 
assignments to the hidden signals. 

Our main technical contribution is a finer complexity analysis of the prob- 
lem, which reveals that each of the two aspects above involves an exponential 
complexity: the first in the number of signals and the second in the size of the 
secret. We start with the need to go over all assignments of hidden signals and 
show that even when the specification is T, the set H of hidden signals is given, 
and there is only one secret, given by a deterministic Biichi automaton, synthesis 
with privacy is EXPTIME-complete. This is exponentially higher than synthesis 
of deterministic Btichi automata, which can be solved in polynomial time. We 
continue to the need to go over all possible choices of H. For that, we focus on 
the closed setting, namely when I = Ø, and the case the specification and secrets 
are given by deterministic automata. We show that while synthesis with privacy 
can be then solved in polynomial time for a given set H, it is NP-complete when 
H is not given, even when the function cost is uniform. 

We continue and study two variants of the problem: bounded synthesis and 
knowledgeable observer. One way for coping with the 2EXPTIME complexity of 
LTL synthesis, which is carried over to a doubly-exponential lower bound on 
the size of the generated system [28], is bounded synthesis. There, the input 
to the problem includes also a bound on the size of the system [30,12,19]. In 
a setting with no privacy, the bound reduces the complexity of LTL synthesis 
to PSPACE, as one can go over all candidate systems. We study bounded syn- 
thesis with privacy and show that privacy makes the problem much harder: it 
is EXPSPACE-complete when the specification and secrets are given by LTL 
formulas, and is PSPACE-complete when they are given by deterministic parity 
(or Biichi) automata. 

Finally, recall that a system keeps a secret w private if an observer cannot 
reveal the truth value of y: every observable computation can be completed both 
to a computation that satisfies ~ and to a computation that does not satisfy w. 
We study a setting in which the observer knows the specification y of the system. 
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Consequently, the observer knows that only completions that satisfy y should 
be taken into account. If, for example, y —> yw, then w cannot be kept private. 
We describe an algorithm for this variant of the problem and analyze the way 
knowledge of the specification influences the complexity. In particular, we show 
that the problem becomes EXPTIME-complete even when the specification is 
given by a deterministic Biichi automaton and the secrets are of a fixed size. 

Due to the lack of space, some examples and proofs are omitted and can be 
found in the full version, in the authors’ URLs. 


2 Preliminaries 


2.1 Synthesis 


For a finite nonempty alphabet X, an infinite word w = 09-0,-:... € X® is an 
infinite sequence of letters from X. A language L C X* is a set of infinite words. 
Let J and O be disjoint finite sets of input and output signals, respectively. 
We consider the alphabet 2740 of truth assignments to the signals in J U O. 
Then, a languages L C (2/¥°)* can be viewed as a specification, and the truth 
value of L in a computation w € (2/¥9)* is T if w € L, and is F otherwise. 

An (I/O)-transducer is a tuple T = (I,O,S,80,7,7), where S is a finite set 
of states, sọ € S is an initial state, ņ : 9 x 27 > S is a transition function, and 
T : S — 2° is a labeling function. We extend the transition function 7 to words 
in (27)* in the expected way, thus 7* : S x (2/)* > S is such that for all s € S, 
xy € (2/)*, andi € 2/, we have that n* (s, €) = s, and n* (s, zr-i) = n(n*(s, xr), i). 
For a word wy = ig: i1 + i2-... € (2), we define the computation of T on wy 
to be the word T(wy) = (io U 09) - (i1 U01) -... € (277°)”, where for all j > 0, 
we have that 0; = T(n*(so,io0---i;)). The language of T, denoted L(T), is the 
set of computations of T, that is L(T) = {T (wr) : wr € (27)*}. 

We say that T realizes a language L C (2/V°)” if L(T) C L. We say that a 
language L C (2'¥°)” is realizable if there is an (I/O)-transducer that realizes 
it. In the synthesis problem, we are given a specification language L C (2/U°)” 
and we have to return an (I/O)-transducer that realizes L or decide that L is 
not realizable. The language L is given by an automaton over the alphabet 2790 
or a temporal logic formula over J U O (see definitions in Section 2.4). 


2.2 Synthesis with privacy 


In the synthesis with privacy problem, we are given, in addition to the specifica- 
tion language Ly C (2/¥°)*, also a secret Ly, C (24¥°)”, which defines a behav- 
ior that we want to hide from an observer”. Thus, we seek an (I/O)-transducer 
that realizes L, without revealing the truth value of Ly in the generated com- 
putations. Keeping the truth value of Ly, secret is done by hiding the truth value 
of some signals in J U O. Before we define synthesis with privacy formally, we 
first need some notations. 


2 See Remark 2.3 for an extension of the setting to multiple and conditional secrets. 
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Consider a set H C TUO of hidden signals. Let V = (IU O) \ H denote 
the set of visible signals. For an assignment o € 2/¥°, let hidez(o) € 2” be 
the restriction of ø to the visible signals. That is, hidez(o)(v) = o(v) for all 
v € V. Also, let noisey(o) C 27YO be the set of assignments that differ from 
o in assignments to the signals in H. Thus, noisey(a) = {o € 2449 : oA 
V = o N V}. Then, for an infinite computation w = co: gp. E (2/7°)%, 
we have that hidez(w) = hide (co) - hidey(o,)--- € (2”)” and noisez(w) is 
the set of all computations that differ from w in assignments to the signals 
in H. Formally, of - o} -+> € noisex(w) iff of € noisez(o;) for all i > 0. Note 
that for all w,w’ € (27¥°)”, it holds that w’ € noisez(w) iff w € noisez(w’) 
iff hidey(w’) = hidez,(w), and that w € noisez(w) for all w € (2/4?) and 
H C IUO. Intuitively, when the signals in H are hidden, then an observer of a 
computation w € (24¥°)” only knows that the computation is in noise (w). 

Consider a specification Lọ C (2/¥°)” and a secret Ly C (2/¥°)”. For a set 
H C IUO of hidden signals, we say that an (1/O)-transducer T H-hides Ly, if 
for all words wy € (2/)”, the truth value of the secret Ly, in the computation 
T(wy) cannot be deduced from hidey(T(wr)). Formally, for every wr € (27)%, 
there exist two computations wt, w~ € noisex(7(wr)), such that wt € Ly and 
w ¢ Ly. We say that T realizes (Ly, Ly, H) with privacy if T realizes Lọ and 
H-hides Ly. We say that (Lo, Ly, H) is realizable with privacy if there exists an 
(1/O)-transducer that realizes (Lọ, Ly, H} with privacy. 

Clearly, hiding is monotone with respect to H, in the sense that the larger 
H is, the more likely it is for an (I/O)-transducer T to H-hide Ly. Indeed, if T 
H-hides Ly, then T H’-hides Ly for all H’ with H C H’. In particular, taking 
H = I U O, we can hide all non-trivial secrets. Hiding of signals, however, is 
not always possible, and may sometimes involve a cost. Formally, we consider 
a hiding cost function cost : I U O — N, which maps each signal to the cost of 
hiding it, and a hiding budget b € N, which bounds the cost that the system may 
use for hiding of signals. The cost of hiding a set H C I UO of signals is then 
cost(H) = >7,cq cost(p), and we say that H respects b if cost(H) < b. Note that 
if cost(p) > b, for p E€ TUO, then p cannot be hidden. Also, when cost(p) = 1 for 
all p € IUO, we say that cost is uniform. Note that then, b bounds the number 
of signals we may hide. 

Now, we say that (Lọ, Ly, cost, b) is realizable with privacy if there exists a 
set H C TUO such that H respects b and (Ly, Ly, H) is realizable with privacy. 
Finally, in the synthesis with privacy problem, we are given Ly, Ly, cost, and b, 
and we have to return a set H C TUO that H respects b and an (1/O)-transducer 
T that realizes (Lo, Ly, H) with privacy, or determine that (Ly, Ly, cost, b) is 
not realizable with privacy. 


2.3 Multiple and conditional secrets 
In this section we discuss two natural extensions of our setting. First, often we 


need to hide form the observer more than one secret. We extend the definition of 
synthesis with privacy to a set of secrets S = {Ly,,Ly,,..., Ly, } in the natural 
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way. Thus, an (I/O)-transducer 7 realizes (py, S, H) with privacy if it realizes y 
and H-hides Ly,, for all i € [k]. Note that 

Then, a conditional secret is a pair (Lg, Ly), consisting of a trigger and a 
secret. The truth value of the secret should be unknown only in computations 
that satisfy the trigger. Formally, for a set H C I U O of hidden signals, we say 
that an (I/O)-transducer T H-hides (Lo, Ly) if for all input sequences wr € 
(2/) such that noisey (T (wr)) C Lo, the truth value of Ly in the computation 
T(wr) cannot be deduced from hidez (T (wr)), thus there exist two computations 
wt, w €E noisey(7T(wr)), such that wt € Ly and wT ¢ Ly. A useful special 
case of conditional secrets is when the trigger and the secret coincide, and so we 
have to hide the truth value of the secret only if there are computations where 
the value of secret is T. Formally, T H-hides (Ly, Lọ) if for all input sequences 
wr € (2/)”, there exists a computation w7 € noisez(T(w7)) such that w~ ¢ Ly. 

Note that unlike a collection of specifications, which can be conjuncted, hid- 
ing a set of secrets is not equivalent to hiding their conjunction. Likewise, hiding 
a conditional secret is not equivalent to hiding the implication of the secret by the 
trigger. Thus, the two variants require an extension of the solution for the case 
of a single or unconditional secret. In Remark 2, we describe such an extension. 


2.4 Automata and LTL 


An automaton on infinite words is A = (X, Q, qo, ô, a), where X is an alphabet, 
Q is a finite set of states, qo € Q is an initial state, 6: Q x X > 2° isa 
transition function, and a is an acceptance condition, to be defined below. For 
states q, s E€ Q and a letter o € X, we say that s is a o-successor of q if s € 6(q,¢). 
Note that we do not require the transition function to be total. That is, we allow 
that ô(q,o) = Ø. If |6(q,o)| < 1 for every state q € Q and letter o € X, then 
A is deterministic. For a deterministic automaton A we view 6 as a function 
6:Qx XY > QU{1L}, where L is a distinguished symbol, and instead of writing 
6(q,a) = {q} and (q, o) = Ø, we write ô(q, o) = q and (q, o) = L, respectively. 

A run of A on w = 009-01-::: E€ &™” is an infinite sequence of states r = ro-r1- 
rg:... E Q”, such that ro = qo, and for all i > 0, we have that ri+ı E€ d(ri, ci). 
The acceptance condition a determines which runs are “good”. We consider 
here the Büchi, co-Btichi, generalized Btichi and parity acceptance conditions. 
All conditions refer to the set inf(r) C Q of states that r traverses infinitely 
often. Formally, inf (r) = {q € Q : q = r; for infinitely many 7’s}. In generalized 
Biichi the acceptance condition is of the form a = {a1,Q2,...,ax}, for k > 1 
and sets a; C Q. In a generalized Biichi automaton, a run r is accepting if for 
all 1 < i < k, we have that inf(r) Na; 4 Ø. Thus, r visits each of the sets 
in a infinitely often. Büchi automata is a special case of its generalized form 
with k = 1. That is, a run r is accepting with respect to the Büchi condition 
a C Q, if inf(r) Na # Ø. Dually, in co-Biichi automata, a run r is accepting 
if inf(r) Qa = Ø. Finally, in a parity automaton, the acceptance condition 
a : Q > {1,..., k}, for some k > 1, maps states to ranks, and a run r is accepting 
if the maximal rank of a state in inf(r) is even. Formally, maxgeing(r) {a(q)} is 
even. A run that is not accepting is rejecting. We refer to the number k in a as 
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the index of the automaton. A word w is accepted by A if there is an accepting 
run of A on w. The language of A, denoted L(A), is the set of words that A 
accepts. Two automata are equivalent if their languages are equivalent. 

We denote the different classes of automata by three-letter acronyms in 
{D,N} x {B,C,GB,P} x {W}. The first letter stands for the branching mode 
of the automaton (deterministic, nondeterministic); the second for the accep- 
tance condition type (Btichi, co-Biichi, generalized Biichi, or parity); and the 
third indicates we consider automata on words. For example, NBWs are nonde- 
terministic Biichi word automata. 

LTL is a linear temporal logic used for specifying on-going behaviors of reac- 
tive systems [24]. Specifying the behavior of (I/O)-transducers, formulas of LTL 
are defined over the set JU O of signals using the usual Boolean operators and 
the temporal operators G (“always”) and F (“eventually”), X (“next time”) 
and U (“until”). The semantics of LTL is defined with respect to infinite com- 
putations in (2/4°)”. Thus, each LTL formula y over J U O induces a language 
Ly C (2/9) of all computations that satisfy y. 

Recall that the input to the synthesis with privacy problem includes lan- 
guages L, and Ly. We sometimes replace Ly and Ly, in the different notations 
with automata or LTL formulas that describe them, thus talk about realizabil- 
ity with privacy of (Ay, Ay, H) or (vy, Y, H), for automata A, and Ay, or LTL 
formulas y and 4%. 


Example 1. Consider a scheduler that serves two users and grant them with 
access to a joint resource. The scheduler can be viewed as an open system with 
I = {req,,reqy}, with req; (i € {1,2}) standing for a request form User i, and 
O = {grant,, grant}, with grant; standing for a grant to User i. The system 
should satisfy mutual exclusion and non-starvation. Formally, the specification 
for the system is 1 \y2A 93, for yı = G((>grant,) V (agrant,)), p2 = G(req, > 
F grant, ), and y3 = G(req, > Fgrant,). 

We may want to hide from an observer of the interaction the exact policy 
scheduling of the system. For example,? the secret pı = ((—grant,)Wreq,) A 
G(grant, > X((>grant,)Wreq, )) reveals whether the system gives User 1 grants 
only after requests that have not been granted yet. Indeed, w specifies that 
once a grant to User 1 is given, no more grants are given to her, unless a new 
request from her arrives. A similar secret can be specified for User 2. Note 
that in order to hide 71, it is sufficient to hide only one of the signals req, 
or grant,. In fact, this is true even when the observer knows the specification 
for the system. Then, the secret Y2 = G((req, —> grant, V Xgrant,) A (reqa > 
grant, V X grant, )) reveals whether delays in grants are limited to one cycle. Here, 
unlike with we, it is not sufficient hiding only a single request or even both. 
Indeed, some policies disclose the satisfaction value of Y2 even when requests 
are hidden. For example, a system that simply alternates between grants, thus 
outputs {grant,}, {grant}, {grant,}, {grant,},..., satisfies the specification and 
clearly satisfies %2 regardless of the users’ requests. 


3 The LTL operator W is “weak Until”, thus pıW p2 = (piUp2) V Gp. 
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Consider now the secret w3 = FG(req, —> grant,), which asserts that even- 
tually, the requests of User 1 are always granted immediately. A system that 
satisfies %3 is unfair to User 2. Aiming to hide this unfair behavior, we can use 
the conditional secret (p3, Y3), which requires a system that satisfies %3 to hide 
its satisfaction. 

Some computations that satisfy %3, however, may still be fair to User 2. 
For example, if %3 is satisfied vacuously or if only finitely many requests are 
sent from User 2, then the behavior specified in %3 is fair, and we need not 
hide it. Accordingly, we can strengthen the trigger w3 and restrict further the 
computations in which the satisfaction value of Y3 should be hidden. Formally, 
we replace the trigger Y3 by a trigger w3/0, for a behavior 0 in which a scheduling 
policy that satisfies w3 is not fair (and hence, need to be hidden). 

Let us consider possible behaviors 6 for the conditional secret (Y3 A 6, 3). As 
discussed above, behaviors that make %3 unfair are GF'req,, implying that Y3 
is not satisfied vacuously, and G'Freq., implying that the immediate grants to 
User 1 are not due to no requests from User 2. Taking 0 = (GF'req,) A (GFreq,) 
results in a more precise conditional secret. 

The trigger 0 can be made more precise: taking 0 = GF (req; A req.) still 
guarantees no vacuous satisfaction and also asserts that immediate grants to 
User 1 are given even when the requests of User 1 arrive together with those 
of User 2. In fact, 0 = GF (req, A (mgrant,)Ureq,)) is even more precise, as 
it excludes the possibility that the requests of User 1 arrive before those of 
User 2. Note that the secret can be made less restrictive too, for example with 
ws = FG(req, > ((-grant,)Ugrant,)), which specifies that eventually, grants to 
User 1 are always given before grants to User 2. 


Example 2. As a different example, consider a paint robot that paints parts of 
manufactured pieces. The set O includes 3 signals co, c1, cz that encode 8 colors. 
The encoding corresponds to the composition of the color from paint in three 
different containers. For example, color 101 stands for the robot mixing paints 
from containers 0 and 2. The observer does not see the generated pattern, but, 
unless we hide it, may see the arm of the robot when it reaches a container. 
Accordingly, hiding of signals in O involve different costs. 

The user instructs the robot whether to stay with the current color or change 
it, thus J = {change}. We seek a system that directs the robot which color to 
chose, in a way that satisfies requirements about the generated pattern. For 
example, in addition to the requirement to respect the changing instructions 
(Erespect), the specification p may require the pattern to start with color 000, 
and if there are infinitely many changes, then all colors are used (£a), yet color 
000 repeats between each two colors (€repear). Formally, 


= erespect = G((X-change) oO ((co oO Xo) mx (cr oO Xc) IN (c2 < Xcə))), 
= Call = GF(@AGAG)AGF(@A G&A c2) A+++ A GF (co Ac, A ca), 

— Erepeat = G((co V c1 V co) + X (change > (ĉ& A & A é))), and 

eS (čo AGA ©) A respect A ((GFchange) > (Eat A repeat ))+ 


We may want to hide from an observer certain patterns that the robot may 
produce. For example, the fact color 111 is used only after color 110 (with color 
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000 between them), the fact there are colors other than 000 that repeat without 
a color different from 000 between them, and more. Note that not all the signals 
in O need to be hidden, and that the choice of signals to hide depends on the 
secrets as well as the cost of hiding. 


3 Solving Synthesis with Privacy 


In this section we describe a solution to the problem of synthesis with privacy 
for LTL specifications and show that it is 2EXPTIME-complete, thus not harder 
than LTL synthesis. The solution is based on replacing the specification by one 
that guarantees the hiding of the secret. For this, we need the following two 
constructions. 


Lemma 1. Consider a nondeterministic automaton A = (2'¥°,Q,qo,6,a). Given 
a set H C IUO, there is a transition function 6% : Q x YO — 22 such 
that the nondeterministic automaton A™ = (2!¥°,Q,qo,6",a) is such that 


L(A*) = noise (L(A)). 


Proof. Intuitively, the transition function 5% increases the nondeterminism of 6 
by guessing an assignment to the signals in H. Formally, for q € Q and a € 2/9, 
we define 6*(q,0) = Usrenoiser(c) 9(4, 0"). It is easy to see that a word w’ is 
accepted by A” iff there is a word w accepted by A such that w’ € noise (w). 


Note that while A maintains the state space and acceptance condition of 
A, it does not preserve determinism. Indeed, unless H = Ø, we have that A% is 
nondeterministic even when A is deterministic. Next, in Lemma 2 we construct 
automata that accept computations that satisfy the specification and hide the 
secret when a given set of signals is hidden. 


Lemma 2. Consider two disjoint finite sets I and O, a subset H C IUO, and 
w-regular languages L, and Ly over the alphabet 2100. There exists a DPW 
DE, with alphabet 2'Y° that accepts a computation w € (2/¥°)¥ iff w € Ly and 
there exist w7, wT € noisey(w) such that wt € Ly and w` ¢ Ly. 


20Cel+lol) 


1. If L, and Ly are given by LTL formulas p and w, then Dey has 2 
states and index 2021+14), 

2. If Ly and Ly are given by DPWs Dy and Dy with ng and ny states, and 
of indices ky and ky, then DH has 20eko (nyky)? log(ne'keny'ky)) states 
and indez O(ng - ky + (ny + ky)?). 


Proof. We start with the case Ly, and Ly are given by LTL formulas y and w. 
Let Ap, Ay and A_,, be NGBWs for Ly, Ly, and Ly. By [31], such NGBWs 
exist, and are of size exponential in the corresponding LTL formulas. Let At 
and A®,, be the NGBWSs for noisex(L(Ay)) and noisey(L(A-w)), respectively, 
constructed as in Lemma 1. 
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Now, let M ay be an NGBW for the intersection of the three automata A,, 
Alt, and AX. The NGBW W can be easily defined on top of the product of the 
three automata, and hence is of size 20(!?!+!4)) and index O(|y| + |w|). Observe 
that indeed, a word w € (2/¥°)” is accepted by Ne, iff w = ọ and there exist 
wt, wT € noisey (w) such that wt = y and w7 j Y. By [29,23], determinizing 
New results in a DPW Dey with 22°"°"*'"? states and index 2021+14), and 
we are done. 

We continue with the case Ly, and Ly are given by DPWs D, and Dy. We 
first obtain from Dy two NBWs, Ay and A~y for Ly and Loy = (244°)* \ Ly 
respectively, and also we translate D, into an equivalent NBW A,. Note that 
the NBWs Ay and A.,, can be defined with O(n, - ky) states, and that A, can 
be defined with O(n, - kọ) states. We then obtain the NBWs AT and AX, by 
applying the construction in Lemma 1 on Ay and A-y, respectively. We then 
define an NBW of size O(ny - ky > (ny + ky)?) for the intersection of the three 
NBWs Ay, A, and A%,,, and finally determinize it into a DPW D%,, with 


QO (no ky: (nyky)” log(ng-ke-rvky)) states and index OGig ko` (nyky)’). 


Remark 1. [The size of De, for specifications and secrets given by 
DBWs| The exponential dependency of Dre in the DPW D, in the construc- 
tion in Lemma 2 follows from the exponential blow up in DPW intersection [4]. 
When Lọ is given by a DBW Dg, we can first construct a DPW for the intersec- 
tion of At and Anis and only then take its intersection with D,. This results 


ina DPW DY T of size exponential in Dy, but only polynomial in Dy. 
We can now solve synthesis with privacy for LTL formulas. 


Theorem 1. [Synthesis with privacy, LTL] Given two disjoint finite sets I 
and O, LTL formulas p and w over I U O, a cost function cost: I U O > N, 
and a budget b € N, deciding whether (p,p, cost, b) is realizable with privacy is 
2EXPTIME-complete. 


Proof. We start with the upper bound. Given y, wv, cost, and b, we go over all 
H C IUO such that cost(H) < b, construct the DPW DH defined in Lemma 2, 
and check whether L(D¥ ,) is realizable. Since realizability of a DPW with n 
states and index k can be solved in time at most O(n*) [5], the 2EXPTIME 
upper bound follows from Des having 22°04 tates and index 2O(lel+I4)), 
For the lower bound, we describe a reduction from LTL synthesis with no 
privacy. Note that adding to a specification y a secret T or F does not work, 
as an observer knows its satisfaction value. It is easy, however, to add a secret 
that is independent of the specification. Specifically, given a specification y over 
IUO, let O' = OU {p}, where p is a fresh signal not in J U O. Consider the 
secret 7 = p and a cost function with cost(p) = 0. Clearly, an (I/O)-transducer 
T realizes ¢ iff the (I/O’)-transducer 7’ that agrees with 7 and always assigns 
F to p, realizes y and {p}-hides 7. Conversely, for an I/O’-transducer 7’, let 
T be the (1/O)-transducer obtained from 7 by ignoring the assignments to p. 
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Clearly, T’ {p}-hides ~. In addition, as y does not refer to p, we have that 7” 
realizes y iff T realizes y. Thus, ¢ is realizable iff (y, Y, cost, 0) is realizable with 
privacy. 


Remark 2. [Solving privacy with multiple and conditional secrets] Recall 
that for a set of secrets S = {y,2,...,Ux}, an ([/O)-transducer T realizes 
(y, S,H) with privacy if it realizes p and H-hides w;, for all i € [k]. It is easy 
to extend Theorem 1 to the setting of multiple secrets by replacing the DPW 
Di, by a DPW obtained by determinizing the product of A, with automata 
At and AX is forall <i<k. 

As for conditional secrets, recall that a computation should satisfy the speci- 
fication, and from the point of view of an observer, either the trigger is not trig- 
gered, thus m € L(A*,), or the secret is hidden, thus 7 € L( AY) N L(A%,)- Ac- 
cordingly, we need to construct a deterministic automaton for L(A,)N(L(A%)U 
L(A) U L(A®,)). This can done by determinizing an NBW that is defined on 
top of the product of Ap, A%,, AX, and AX. 


While the complexity of our algorithm is not higher than that of LTL syn- 
thesis with no privacy, it would be misleading to state that handling of privacy 
involves no increase in the complexity. Indeed, the algorithm involved two com- 
ponents whose complexity may have been dominated by the doubly exponential 
translation of the LTL formulas to deterministic automata: 


1. A need to go over all candidate sets H C TUO. 
2. A need to check that the generated transducer H-hides the secret. 


In the next two sections, we isolate these two components of synthesis with 
privacy and show that each of them involves an exponential complexity: the first 
in the number of signals and the second in the size of the secret. 


3.1 Hiding secrets is hard 


The synthesis problem for DBWs can be solved in polynomial time. Indeed, the 
problem can be reduced to solving a Biichi game played on top of the specification 
automaton. In this section we show that synthesis with privacy is EXPTIME 
hard even for a given set H of hidden signals (in fact, even a singleton set H C I), 
a trivial specification, and a secret given by a DBW. 

We start by showing that H-hiding is hard even for secrets given by DBWs. 


Theorem 2. Given two disjoint finite sets I and O, a DBW Dy over 24°, 
and a set H C IUO of hidden signals, deciding whether there exists an (I/O)- 
transducer that H-hides Dy is EXPTIME-hard. The problem is EXPTIME-hard 
already when H C I. 


Proof. We describe a polynomial-time reduction from NBW realizability, which 
is EXPTIME-hard [26,17]. Given an NBW A over 2/9, we define a set of signals 
H and a DBW Dy over 2/¥9°%, such that L(A) is realizable iff there exists an 
((I UH)/O)-transducer that H-hides L(D,). 
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Let A = (2/49, Q, qo, 6, a). W.1.0.g, we assume that A has a single initial state 
and that every word in (2//°)” has at least one rejecting run in A. The latter 
can be achieved, for example, by adding a nondeterministic transition from the 
initial state to a rejecting sink upon any assignment iUo € 2/¥°. Let H be a set of 
signals that encode Q. Thus, each assignment s € 2” is associated with a single 
state in Q. We refer to a letter in 2770Y% as a pair (ø, q) € 244° x Q, and we view 
a word in (2/U0U#)” as the combination w ® r, of a word w € (2/7?)” with a 
word r € Q”. Formally, for w = 09-01 -+ € (2/¥9)” and r = r1 r2- € QY, let 
wr = (90,71) + (01, 72) +++ E€ (2170H w. Then, we define Dy so that L(Dy) = 
{w r e (2!U0U#)” : the sequence qo - r is an accepting run of A on w}. Note 
that since every word in (2/¥°)” has at least one rejecting run in A, then every 
word w € L(A) has at least one word rt € Q“ such that w @rt € L(Dy,) and 
at least one word r~ € Q“ such that w 9 r7 ¢ L(Dy). 

Formally, Dy = (240°, Q,qo,6’,a) has the same state space and accep- 
tance condition as A, and it uses the Q@-component of each letter in order 
to resolve the nondeterministic choices in A. Thus, the transitions function 
6’: Q x QlvOUH _, Q is defined as follows. For every state q € Q and letter 
(o,s) € 2/UOUH we have that 6’(q,(c,s)) = s if s € 6(q,c), and otherwise 
6'(q, (o,8)) = L. We prove that indeed L(D,,) accepts exactly all words w @ r 
such that qo: r is an accepting run of A on w. By definition of 6’, it holds that 
r’ is a run of Dy over w @r iff r’ = qo- r, and qor is a run of A over w. Hence, 
a run r’ = qo -r of Dy over w @r is accepting, iff inf (r) Na A Ø, iff r’ = qor 
is an accepting run of A over w, and we are done. 


Note that in the proof of Theorem 2, we could have defined H so that it 
resolves the nondeterminism in A in a more concise way. In particular, if we 
assume that the nondeterminism degree in A is at most 2, then a set H of size 
1 can resolve the nondeterminizm of 6. Hence, as NBW synthesis is EXPTIME- 
hard already for NBWs with branching degree 2 (this follows from the fact 
that a bigger branching degree can be decomposed along several transitions), 
EXPTIME hardness holds already when hiding a single input signal. 


Theorem 3. [Synthesis with privacy, DPWs] Given two disjoint finite sets 
I and O, DPWs Do and Dy over 2!Y°, and a set H C IUO of hidden signals, 
deciding whether (Do, Dy, H) is realizable with privacy is EXPTIME-complete. 
Moreover, hardness holds already when the specification is trivial and the secret 
is given by a DBW. 


Proof. For the upper bound, we solve the synthesis problem for the DPW Di, 
defined in Lemma 2. As specified there, the size of DE is exponential in the size 
and index of both D, and Dy, and its index is polynomial in the size and index 
of D, and Dy. Membership in EXPTIME then follows from the complexity of 
the synthesis problem for DPWs [2]. 

For the lower bound, fix a DBW Dy such that L(Dr) = (2/¥°)*. Then, it 
is easy to see that (Dr, Dy, H} is realizable with privacy iff there is an (I/O)- 
transducer that H-hides Dy. Thus, hardness in EXPTIME follows from Theo- 
rem 2. 
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3.2 Searching for a set of signals to hide is hard 


Another component in the algorithm that is dominated by the doubly-exponential 
translation of LTL to DPWs is the need to go over all subsets of [UO in a search 
for the set H of signals to hide. Trying to isolate the influence of this search, it 
is not enough to consider specifications and secrets that are given by DBWs, as 
the synthesis with privacy problem is EXPTIME-hard already for a given set 
H, and so again, the complexity of the search is dominated by the complexity 
of the synthesis problem. Fixing the size of the secret, which is the source of the 
exponential complexity, does not not work either, as it also fixes the number of 
signals that we may need to hide. We address this challenge by moving to an 
even simpler setting for the problem, namely synthesis with privacy of a closed 
system. We are going to show that in this setting, the search for H is the only 
non-polynomial component in the algorithm. 

In the closed setting, all signals are controlled by the system, namely I = 
@. Consequently, each transducer has a single computation, and realizability 
coincides with satisfiability. In particular, for J = Ø, we have that (Ly, Ly, H} 
is realizable with privacy iff there exists a word w € Ly, for which there exist 
two words wt, w~ € noisey (w) such that wt € Ly and w~ ¢ Ly. We show that 
while synthesis with privacy in the closed setting can be solved in polynomial 
time for a given set H of hidden signals, it is NP-complete when H is not given, 
even when the function cost is uniform. 

We start with the case H is given. 


Theorem 4. Given a finite set O of output signals, a set H C O of hidden sig- 
nals, and DPWs D, and Dy over 2°, deciding whether (Dy, Dy, H) is realizable 
with privacy can be done in polynomial time. 


Proof. First, we complement Dy, which results in a DPW D-y of the same 
size, and of index k + 1, where k is the index of Dy. Then, we translate Dy, 
Dy and D.y into equivalent NBWs Ap, Ay and A y, respectively. All three 
NBWs can be defined in size that is polynomial in their deterministic DPW 
counterpart. Let At and AH, be NBWs obtained by applying the construction 
in Lemma 1 on Ay and A-y, respectively. By Lemma 1, the NBWs At and 
At, have the same number of states as Ay and Ay respectively. Let N be 
an NBW for the intersection L(A,) N L(A7) N L(A%,,). Note that M can be 
defined with size that is polynomial in Ay, AG and Ane Moreover, M accepts a 
wore w iff w € L(A,), and there exist two words wt, w7 € noisez(w) such that 

+ € L(A,y) and wT ¢ L(A,). Thus, realizability with privacy of (Ay, Ay, H) 
can be reduced to the nonemptiness of M, which can be decided in polynomial 
time. 


We continue to the case H should be searched. 


Theorem 5. Given a finite set of output signals O, DPWs A, and Ay over 
2°, a hiding cost function cost : O — N, and a budget b € N, deciding whether 
(Ay, Ay, cost, 6) is realizable with privacy is NP-complete. Moreover, hardness 
holds already when the specification and secret are given by DBWs. 
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Proof. For the upper bound, a nondeterministic Turing machine can guess a set 
H C O, check whether cost(H) < b, and, by Theorem 4, check in polynomial 
time whether (Ap, Ay, H) is realizable with privacy. 

For the lower bound, we describe a polynomial-time reduction from the 
vertex-cover problem. In this problem, we are given an undirected graph G = 
(V, E) and k > 1, and have to decide whether there is a set S C V such that 
|S| < k and for every edge {v,u} € E, we have that E N S # Ø. Given an 
undirected graph G = (V, E}, with E = {e1,e2,...,@m}, we consider a closed 
setting with O = V and construct DBWs A, and Ay over the alphabet 2V 
such that for all H C V, it holds that (Ap, Ay, H) is realizable with privacy 
iff H is a vertex cover of G. Accordingly, there is a vertex cover of size k in G 
iff (Ap, Ay, cost, k} is realizable with privacy for the uniform cost function that 
assigns 1 to all signals in O. 

We define A, and Ay over the alphabet 2V as follows. The DBW Ay is a 
2-state DBW that accepts the single word Ø“. The DBW Ay = (2”,Q,u,6, a) 
for the secret is defined as follows. The set of states is Q = {q1,q2,---;@m+1}; 
the set of accepting states is a = {qm+1 } and the transition function 6 is defined 
for all S C O andi < m by, ôlqi, S) = q4i if SNe; A Ø, and d(q,S) = L 
otherwise. Finally, 6(¢m+1,5) = qm+ı for all S C V. That is, words in L(A,y) 
encode vertex covers of G. Indeed, if w = S1 -S2-S2-...€ L(Ay), then for all 
i < m we have that S; N e; 4 Ø. Thus, if for all i < m we set v; E€ V to be some 
vertex in S; N e;, then we get that {v;,...,Um} is a vertex cover of G. 

In the full version, we prove that (Ap, Ay, H) is realizable with privacy iff H 
is a vertex cover of G. 


4 Bounded Synthesis with Privacy 


In the general synthesis problem, there is no bound on the size of the generated 
system. It is not hard to see that if a system that realizes the specification 
exists, then there is also one whose size is bounded by the size of a deterministic 
automaton for the specification. For the case of LTL specifications, this gives 
a doubly-exponential bound on the size of the generated transducer, which is 
known to be tight [28]. In [30], the authors suggested to study bounded synthesis, 
where the input to the problem includes also a bound on the size of the system. 
The bound not only guarantees the generation of a small system, if it exists, 
but also reduces the complexity of the synthesis problem and gives rise to a 
symbolic implementation and further extensions [12,19]. In particular, for LTL, 
it is easy to see that bounded synthesis can be solved in PSPACE, as one can 
go over and model-check all candidate systems. For specifications in DPW, the 
bound actually increases the complexity, as going over all candidates results in 
an algorithm in NP. 

In this section we study bounded synthesis with privacy. As in traditional 
synthesis, the hope is to both reduce the complexity of the problem and to end 
up with smaller systems. In addition to a specification Lọ, a secret Ly, and a 
set H C I UO of hidden signals, we are given a bound n € N, represented in 
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unary, and we are asked to construct an (J/O)-transducer with at most n states 
that realizes (Lo, Ly, H) with privacy, or to determine that no such transducer 
exists. As in the unbounded case, we can define the problem also with respect 
to a hiding cost function and a budget. 


4.1 Hiding secrets by a bounded system is hard 


We first show that hiding secrets in a bounded setting is hard. In fact, the 
complexity of hiding goes beyond the complexity of bounded synthesis with 
no privacy already in the case the specification and secrets are given by LTL 
formulas. In the case of DBWs and DPWs, hiding is also more complex than 
bounded synthesis without privacy, but the difference is not significant. 

The key idea in both results is similar to the one in the proof of Theorem 2. 
There, we reduce realizability of NBWs to hiding of secrets given by DBWs. 
Essentially, we use the hidden signals to imitate nondeterminism. Here, with 
a bound on the size of the system, we cannot reduce from realizability, as the 
problem has the flavor of model checking many candidates. Accordingly, we 
reduce from universality, either in the form of LTL formulas with universally- 
quantified atomic propositions, or in the form of language-universality for NBWs. 


Theorem 6. Given two disjoint finite sets I and O, an LTL formula w over 
2100 a set HC IUO of hidden signals, and a bound n > 1, given in unary, 
deciding whether there exists an (I/O)-transducer of size at most n that H-hides 
w is EXPSPACE-hard. The problem is EXPSPACE-hard already when n = 1. 


Theorem 7. Given two disjoint finite sets I and O, a DBW Dy over geno. 
a set H C IUO of hidden signals, and a bound n > 1, represented in unary, 
deciding whether there exists an (I/O)-transducer of size at most n that H-hides 
Dy is PSPACE-hard. The problem is PSPACE-hard already when n = 1. 


4.2 Solving bounded synthesis with privacy 


We can now present the tight complexity for bounded synthesis with privacy 
for both types of specification formalisms. For the upper bounds, we construct 
an NGBW M, ine that accepts exactly all words that satisfy y and hide w, and 
search for an (I/O)-transducer of size n whose language is contained in that of 
Ni. 

Theorem 8. [Bounded synthesis with privacy] Given two disjoint finite 
sets I and O, specification Ly and secret Ly, over TUO, a set H C IUO of hidden 
signals, and a bound n € N, represented in unary, deciding whether there is an 
(1/O)-transducer with at most n states that realizes (Lo, Ly, H) with privacy is 
PSPACE-complete for Ly and Ly given by DPWs, and is EXPSPACE-complete 
for Lg and Ly given by LTL formulas. Hardness in PSPACE holds already for 
DBWs. 
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5 When the Observer Knows the Specification 


In this section we study a setting in which the observer knows the specification 
y of the system. Technically, it means that when the observer tries to evaluate 
the secret, she knows that only computations that satisfy y should be taken into 
account. If, for example, y > w, then w= cannot be kept private in a setting in 
which the observer knows y. Indeed, the fact y is realized by the system reveals 
that w is satisfied. Formally, we say that T realizes (p, Y, H} with privacy under 
the knowledge of the specification if T realizes p, and for every wr € (2/)”, there 
exist wt, w~ € noisey (T (wr)) N Ly such that wt = y and w7 jÆ y. Thus, the 
satisfaction of the secret y in a computation J (wz) cannot be deduced from the 
observable computation hide (7 (wzr)) even when the observer knows that ¢ is 
satisfied in 7 (wz). The adjustment for the definition of the problem with respect 
to a hiding cost function and a budget is similar. 

We start by showing the analogue of Lemma 2 for the setting in which the 
observer knows the specification. The construction is similar to that of Lemma 2, 
except that now, the construction of the DPW Di involves an existential pro- 
jection on H also in the automaton for the specication. Accordingly, the size 
of the DPW is exponential in both the specification and the secret even in the 
case they are given by DBWs. 


Lemma 3. Consider two disjoint finite sets I and O, a subset H C IUO, and 
regular languages Ly, and Ly over the alphabet 2100. There exists a DPW Div. 
with alphabet 2'V° that accepts a computation w € (2/V°)* iffw € Ly and there 
exist wt, w E noisey (w) such that wt € Lọ N Ly and wT € Ly \ Ly. 


1. If L, and Ly are given by LTL formulas p and w, then Dho has 22007141"? 
states and index 2021+14) , 

2. If Ly and Ly are given by DPWs Dy and Dy with ny and ny states, and of 
indices ky and ky, then Dio has 20 (Meko)? (nyky)? log(ng-ke-nwkv)) states 
and indez O((ng + ky)? + (ny + ky)?). 


Lemma 3 implies that all the asymptotic upper bounds described in Section 3 
are valid also in a setting with an observer that knows the specification. Also, 
as the lower bounds in Theorems 1 and 3 involve secrets that are independent 
of the specification, they are valid for this setting too. Two issues require a 
consideration: 


1. The need to search for H: the NP-hardness proof in Theorem 5 is no longer 
valid, as there, p — ~y, and so the satisfaction value of the secret is revealed 
in a setting with an observer that knows the specification. 

2. The construction in Lemma 3 results in an algorithm that is exponential 
also in the specification, even when given by a DBW. On the other hand, 
the EXPTIME-hardness proof in Theorem 2 does not imply an exponential 
lower bound in the specification. 
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Below we address the two issues, providing lower bounds for a setting in which 
the observer knows the specification. Matching upper bounds follow the same 
considerations in Theorems 3 and 5, where Di . replaces Dia We start with a 
variant of Theorem 5, showing NP-hardness also in the setting of a knowledgeable 
observer. As mentioned above, the lower bound in the proof of Theorem 5 does 
not work when the observer knows the specification, yet, can easily be modified 
to work for the case of a knowledgeable observer. 


Theorem 9. Given a set O of output signals, a cost function cost: O > N, a 
hiding budget b € N, and DBWs A, and Ay over 2°, deciding whether there is 
H C O, with cost(H) < b, such that (Ay, Ay, H) is realizable with privacy under 
knowledge of the specification is NP-hard. Moreover, hardness holds already when 
cost is uniform. 


We continue to the second issue, proving that synthesis with privacy under 
knowledge of the specification is EXPTIME-hard even for specifications in DBWs 
and secrets of a fixed size. Note that synthesis with privacy (without knowledge 
of the specification) can be solved in PTIME in this case (see Remark 1). The 
proof is similar to that of Theorem 2, except that here the lower bound needs the 
secret to be a of a fixed size, making the specification more complex. It follows 
that the exponential blow-up in D,, which exists in Lemma 3 cannot be avoided 
even when it is a DBW and Dy is of a fixed size. 


Theorem 10. Given two disjoint finite sets I and O, DBWs Dy and Dy over 
210, and a set H C IUO of hidden signals, deciding whether (Dp, Dy, H) is 
realizable with privacy under knowledge of the specification is EXPTIME-hard 
already when Dy is of fixed size. 


Remark 3. Recall that an observer that knows the specification can restrict the 
search for computations on which she evaluates the secret to ones that satisfy 
the specification. In fact, the observer can do better, and restricts the search 
to computations that are generated by an (J/O)-transducer that realizes the 
specification. 

In order to see the difference between the two definitions, consider the case 
where I = H = {p1, po}, O = {q}, y = (q © pı) V Gpo, and w = pı. An observer 
that knows y does not know which of its two disjuncts is satisfied, and thus, 
even though she observes q, the value of pı stays secret. Formally, a transducer 
that realizes q © pı H-hides y% from the observer, even if the observer knows 
that ọ is satisfied. Indeed, for every observable computation x € 2, there 
is a computation wt € noisey(K«) that satisfies pı A Gp2 and a computation 
wT € noisey(K) that satisfies (~p1) A Gp2. Hence, (p, Y, H) is realizable with 
privacy even when the observer knows the specification. 

On the other hand, a clever observer, especially one that has read [16,14], 
knows that a transducer 7 realizes y iff T realizes q + pı. Indeed, if T does not 
satisfy q + pı, then y is not satisfied in computations that do not satisfy G'po, 
which is the case for almost all the computations of T. Accordingly, a clever 
observer that knows y can learn the secret pı by observing the value of q. 
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Using the terminology of [16,14], the specification y and q © pı are open 
equivalent: for every transducer 7, we have that 7 realizes ọ iff T realizes q © pı. 
Note that open equivalence is weaker than equivalence. Once we can simplify a 
specification to an open-equivalent specification that does not include inherent 
vacuity, the two definitions coincide. Such a simplification, however, requires 
further study. Also, as an unrealizable specification is open-equivalent to F, such 
a simplification is at least as complex as the realizability problem (which is also 
good news, as it means that an observer needs to solve a 2EXPTIME problem 
in order to benefit from the difference between the definitions). 


6 Directions for Future Research 


We suggested a framework for the synthesis of systems that satisfy their spec- 
ifications while keeping some behaviors secret. Behaviors are kept secret from 
an observer by hiding the truth value of some input and output signals, subject 
to budget restrictions: each signal has a hiding cost, and there is a bound on 
the total hiding cost. Our framework captures settings in which the choice and 
cost of hiding are fixed throughout the computation. For example, settings with 
signals that cannot be hidden (e.g., alarm sound, or the temperature outside), 
signals that can be hidden throughout the computation with some effort (e.g., 
hand movement of a robot), or signals that are anyway hidden (e.g., values of 
internal control variables). Our main technical contribution are lower bounds for 
the complexity of the different aspects of privacy: the need to choose the hidden 
signals, and the need to hide the secret behaviors. We show that both aspects 
involve an exponential blow up in the complexity of synthesis without privacy. 


The exponential lower bounds apply already in the relatively simple cost 
mechanism we study. Below we discuss possible extensions of this mechanism. 
In settings with a dynamic hiding of signals, we do not fix a set H C IU 
O of hidden signals. Instead, the output of the synthesis algorithm contains a 
transducer that describes not only the assignments to the output signals but 
also the choice of input and output signals that are hidden in the next cycle of 
the interaction. Thus, signals may be hidden only in segments of the interaction 
— segments that depend on the history of the interaction so far. For example, we 
may hide information about a string that is being typed only after a request for 
a password. In addition, the cost function need not be fixed and may depend on 
the history of the interaction too. For example, hiding the location of a robot may 
be cheap in certain sections of the warehouse and expensive in others. Solving 
synthesis with privacy in a setting with such dynamic hiding and pricing of 
signals involves automata over the alphabet 3/V°, reflecting the ability of signals 
to get an “unknown” truth value in parts of the computation. Moreover, as the 
cost is not known in advance (even when the cost of hiding signals is fixed), 
several mechanisms for bounding the budget are possible (energy, mean-payoff, 
etc. [3,6]). 
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